-- Logs begin at Thu 2022-10-13 12:30:34 UTC, end at Thu 2022-10-13 13:29:44 UTC. -- Oct 13 12:56:16.792165 dv-cinder-826607-8-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 13 12:56:16.800287 dv-cinder-826607-8-iscsi devstack@keystone.service[81505]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 13 12:56:16.802303 dv-cinder-826607-8-iscsi devstack@keystone.service[81505]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Oct 13 12:56:16 2022] *** Oct 13 12:56:16.802303 dv-cinder-826607-8-iscsi devstack@keystone.service[81505]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 13 12:56:16.802303 dv-cinder-826607-8-iscsi devstack@keystone.service[81505]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 13 12:56:16.802303 dv-cinder-826607-8-iscsi devstack@keystone.service[81505]: nodename: dv-cinder-826607-8-iscsi Oct 13 12:56:16.802303 dv-cinder-826607-8-iscsi devstack@keystone.service[81505]: machine: x86_64 Oct 13 12:56:16.802303 dv-cinder-826607-8-iscsi devstack@keystone.service[81505]: clock source: unix Oct 13 12:56:16.802696 dv-cinder-826607-8-iscsi devstack@keystone.service[81505]: pcre jit disabled Oct 13 12:56:16.802696 dv-cinder-826607-8-iscsi devstack@keystone.service[81505]: detected number of CPU cores: 4 Oct 13 12:56:16.802696 dv-cinder-826607-8-iscsi devstack@keystone.service[81505]: current working directory: / Oct 13 12:56:16.802696 dv-cinder-826607-8-iscsi devstack@keystone.service[81505]: detected binary path: /usr/bin/uwsgi-core Oct 13 12:56:16.802696 dv-cinder-826607-8-iscsi devstack@keystone.service[81505]: your processes number limit is 47680 Oct 13 12:56:16.802696 dv-cinder-826607-8-iscsi devstack@keystone.service[81505]: your memory page size is 4096 bytes Oct 13 12:56:16.802696 dv-cinder-826607-8-iscsi devstack@keystone.service[81505]: detected max file descriptor number: 2048 Oct 13 12:56:16.802696 dv-cinder-826607-8-iscsi devstack@keystone.service[81505]: lock engine: pthread robust mutexes Oct 13 12:56:16.802696 dv-cinder-826607-8-iscsi devstack@keystone.service[81505]: thunder lock: enabled Oct 13 12:56:16.802696 dv-cinder-826607-8-iscsi devstack@keystone.service[81505]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 13 12:56:16.803340 dv-cinder-826607-8-iscsi devstack@keystone.service[81505]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 13 12:56:16.842878 dv-cinder-826607-8-iscsi devstack@keystone.service[81505]: Python main interpreter initialized at 0x5607ddf419c0 Oct 13 12:56:16.842878 dv-cinder-826607-8-iscsi devstack@keystone.service[81505]: python threads support enabled Oct 13 12:56:16.842878 dv-cinder-826607-8-iscsi devstack@keystone.service[81505]: your server socket listen backlog is limited to 100 connections Oct 13 12:56:16.842878 dv-cinder-826607-8-iscsi devstack@keystone.service[81505]: your mercy for graceful operations on workers is 90 seconds Oct 13 12:56:16.843314 dv-cinder-826607-8-iscsi devstack@keystone.service[81505]: mapped 403077 bytes (393 KB) for 2 cores Oct 13 12:56:16.843428 dv-cinder-826607-8-iscsi devstack@keystone.service[81505]: *** Operational MODE: preforking *** Oct 13 12:56:16.843703 dv-cinder-826607-8-iscsi devstack@keystone.service[81505]: *** uWSGI is running in multiple interpreter mode *** Oct 13 12:56:16.843703 dv-cinder-826607-8-iscsi devstack@keystone.service[81505]: spawned uWSGI master process (pid: 81505) Oct 13 12:56:16.843906 dv-cinder-826607-8-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 13 12:56:16.845860 dv-cinder-826607-8-iscsi devstack@keystone.service[81505]: spawned uWSGI worker 1 (pid: 81520, cores: 1) Oct 13 12:56:16.845860 dv-cinder-826607-8-iscsi devstack@keystone.service[81505]: spawned uWSGI worker 2 (pid: 81521, cores: 1) Oct 13 12:56:16.845860 dv-cinder-826607-8-iscsi devstack@keystone.service[81505]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 13 12:56:21.081832 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 13 12:56:21.081832 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: from cryptography.utils import int_from_bytes Oct 13 12:56:21.082624 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 13 12:56:21.082624 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: from cryptography.utils import int_from_bytes Oct 13 12:56:21.082764 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 13 12:56:21.082764 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: from cryptography.utils import int_from_bytes Oct 13 12:56:21.083219 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 13 12:56:21.083219 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: from cryptography.utils import int_from_bytes Oct 13 12:56:21.131321 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 13 12:56:21.131466 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 13 12:56:21.131537 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 13 12:56:21.131666 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 13 12:56:21.131787 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] command line args: [] {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 13 12:56:21.131865 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] command line args: [] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 13 12:56:21.131947 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 13 12:56:21.132093 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ================================================================================ {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 13 12:56:21.132190 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 13 12:56:21.132268 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ================================================================================ {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 13 12:56:21.132338 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] admin_token = **** {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.132501 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] admin_token = **** {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.132584 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] config_dir = [] {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.132732 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.132815 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] config_dir = [] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.132910 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] config_source = [] {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.132993 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.133086 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] debug = True {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.133169 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] config_source = [] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.133247 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] debug = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.133323 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.133518 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.133668 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.133757 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.133823 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.133880 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.134037 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.134141 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.134278 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.134390 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.134487 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] list_limit = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.134586 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] list_limit = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.134686 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] log_config_append = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.134791 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] log_config_append = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.134885 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.134989 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.135122 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] log_dir = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.135236 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] log_dir = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.135337 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] log_file = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.135431 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] log_file = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.135536 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.135678 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.135767 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.135918 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.136001 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.136072 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.136137 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.136256 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.136373 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.136448 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.136561 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.136641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.136721 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.136841 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.136915 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.137027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.137114 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.137235 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.137319 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.137389 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.137502 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.137581 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.137692 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.137774 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.137927 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.138050 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.138170 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.138281 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.138406 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.138530 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.138882 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.139006 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.139143 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] publish_errors = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.139266 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] publish_errors = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.139387 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.139497 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.139607 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.139724 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.139847 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.139965 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.140077 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.140204 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.140325 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.140439 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.140550 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] standard_threads = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.140660 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] standard_threads = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.140777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.140896 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.141017 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.141135 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.141253 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.141484 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.142114 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] use_journal = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.142233 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] use_journal = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.142421 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] use_json = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.142542 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] use_json = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.142657 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] use_stderr = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.142770 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] use_stderr = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.142890 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] use_syslog = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.143004 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] use_syslog = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.143147 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.143264 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 13 12:56:21.143384 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.143500 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.143620 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.143734 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.143856 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.143970 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.144096 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.144217 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.144338 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.144458 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.144586 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.144707 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.144821 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.144934 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.145047 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.145166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.145289 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.145403 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.145516 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.145630 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.145767 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.145889 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.146016 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.146121 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.146238 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.146351 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.146464 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.146576 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.146698 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.146819 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.146940 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.147090 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.147208 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.147331 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.147456 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.147577 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.147698 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.147812 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.147931 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.148122 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.148245 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.148358 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.148471 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.148584 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.148696 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.148818 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.148933 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.149046 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.149159 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.149262 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.149373 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.149494 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.149609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.149721 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.149834 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.149946 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.150065 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.150191 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.150306 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.150424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.150544 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.150658 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.150771 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.150892 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.151007 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.151131 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.151245 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.151357 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.151474 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.151599 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.151715 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.151827 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.151940 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.152051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.152170 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.152293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.152408 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.152520 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.152633 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.152750 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.152864 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.152986 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.153131 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.153245 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.153358 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.153469 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.153587 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.153775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.153889 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.154001 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.154113 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.154224 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.154336 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.154456 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.154570 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.154681 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.154793 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.154903 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.155015 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.155182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.155297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.155408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.155520 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.155637 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.155736 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.155840 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.155936 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.156031 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.156126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.156227 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.156323 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.156427 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.156523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.156627 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.156756 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.156880 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.157003 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.157118 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.157230 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.157341 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.157453 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.157565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.157676 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.157788 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.157901 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.158015 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.158141 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.158256 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.158382 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.158497 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.158613 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.158725 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.158838 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.158976 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.159112 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.159228 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.159354 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.159465 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.159577 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.159696 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.159806 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.159918 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.160039 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.160149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.160271 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.160271 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.160473 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.160590 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.160712 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.160818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.160928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.161039 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.161151 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.161263 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.161384 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.161500 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.161619 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.161733 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.161844 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.161956 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.162080 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.162194 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.162305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.162418 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.162529 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.162641 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.162762 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.162876 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.162987 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.163138 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.163253 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.163365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.163491 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.163606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.163718 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.163830 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.163942 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.164054 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.164175 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.164289 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.164407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.164522 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.164633 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.164745 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.164937 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.165053 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.165172 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.165286 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.165407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.165531 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.165651 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.165774 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.165887 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.165999 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.166112 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.166230 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.166343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.166463 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.166578 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.166690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.166802 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.166914 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.167033 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.167174 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.167297 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.167413 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.167526 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.167640 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.167753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.167875 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.167996 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.168120 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.168241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.168383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] auth.external = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.168508 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.168624 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.168739 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] auth.external = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.168739 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.168964 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.168964 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.169179 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.169303 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] auth.password = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.169418 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.169542 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] auth.token = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.169657 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] auth.password = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.169773 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.169887 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] auth.token = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.170001 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.170116 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.170242 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.170357 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.170472 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.170587 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.170702 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.170817 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.170940 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.171079 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.171198 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.171313 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.171427 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] credential.caching = True {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.171542 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.171670 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.171786 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] credential.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.171899 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.172012 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.172125 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.172237 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.172359 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.172473 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.172599 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.172713 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.172826 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.172939 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.173061 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.173175 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.173287 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.173406 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.173520 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.173633 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.173755 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.173870 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.173984 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.174095 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.174198 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.174310 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.174432 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.174547 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.174659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.174772 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.174885 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.174998 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.175140 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] federation.caching = True {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.175257 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.175371 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.175483 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] federation.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.175596 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.175708 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.175835 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.175950 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.176065 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.176178 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.176291 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.176414 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.176529 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.176642 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.176762 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.176882 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.176997 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.177123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.177239 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.177360 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.177484 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.177598 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.177711 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.177842 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.177957 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] identity.caching = True {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.178069 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.178183 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.178296 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.178408 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.178596 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.178713 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.178827 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.178940 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.179098 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.179216 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.179346 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.179462 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.179574 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.179687 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.179799 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.179912 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.180034 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.180149 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.180263 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.180382 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.180496 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.180609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.180730 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.180845 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.180964 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.181079 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.181192 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.181306 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.181427 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.181548 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.181662 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.181776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.181891 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.182005 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.182132 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.182255 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.182371 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.182486 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.182600 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.182714 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.182838 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.182954 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.183085 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.183204 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.183319 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.183434 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.183552 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.183667 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.183781 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.183895 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.184009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.184123 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.184249 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.184366 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.184481 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.184595 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.184708 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.184822 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.184945 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.185069 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.185185 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.185299 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.185413 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.185527 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.185650 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.185766 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.185887 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.186008 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.186123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.186246 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.186363 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.186478 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.186592 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.186706 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.186820 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.186942 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.187111 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.187230 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.187344 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.187458 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.187573 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.187699 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.187816 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.187931 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.188044 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.188651 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.188770 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.188893 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.188893 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.189108 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.189108 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.189574 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.189689 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.189812 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.189928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.190043 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.190043 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.190256 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.190370 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.190493 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.190609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.user = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.190724 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.190838 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.190952 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.190952 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.191204 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.191322 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.191436 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.191554 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.191668 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.191781 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.191903 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.192018 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.192131 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.192131 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.192342 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.192455 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.192577 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.192692 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.192806 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.192918 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.193030 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.193143 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.193265 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.193380 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.193493 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.193605 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.193718 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.193839 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.193963 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.194077 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.194190 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.194304 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.194416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.194528 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.194650 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.194765 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.194879 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.194992 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.195133 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.195250 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.195377 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.195494 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.195608 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.195722 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.195837 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.195952 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.196074 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.196190 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.196305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.196425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.196541 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.196655 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.196778 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.196894 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.197009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.197124 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.197238 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.197351 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.197474 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.197590 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.197704 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.197826 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.197941 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.198056 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.198179 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.198295 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.198409 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.198523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.198637 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.198751 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.198873 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.198989 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.199126 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.199244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] resource.caching = True {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.199358 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.199479 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.199603 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.199719 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.199833 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] resource.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.199947 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.200062 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.200177 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.200367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.200484 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.200598 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.200712 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.200826 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.200939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.201063 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.201180 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.201548 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.201669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.201784 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.201972 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] role.caching = True {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.202090 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.202207 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] role.driver = sql {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.202321 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.202435 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.202549 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] role.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.202671 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.202787 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] role.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.202901 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.203016 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.203174 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.203290 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.203418 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.203534 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.203650 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.203764 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.204484 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.204603 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.204728 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.204845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.204960 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.205074 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.205188 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.205303 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.205426 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.205542 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.205656 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.205770 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.205883 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.205998 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.206120 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.206236 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.206236 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.206453 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.206453 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.206667 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.206791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.206897 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.207010 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.207139 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.207253 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.207367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.207496 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.207612 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.207726 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.207840 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.207954 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.208077 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.208193 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.208307 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.208421 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.208536 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.208650 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.208773 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.208889 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.209004 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.209118 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.209232 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.209346 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.209468 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.209585 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.209699 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.209813 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.209927 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.210041 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.210163 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.210279 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.210393 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.210507 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.210621 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.210734 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] token.caching = True {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.210857 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.210973 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.211134 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.211251 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.211365 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] token.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.211478 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.211641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.211762 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.211876 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.211990 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.212104 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.212218 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.212340 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.212457 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.212571 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.212686 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.212799 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.212913 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.213035 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.213151 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.213266 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.213380 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.213531 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.213648 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.213772 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.213889 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.214003 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.214116 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.214230 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.214344 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.214471 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.214600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.214716 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.214831 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.214945 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.215086 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] database.connection = **** {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.215213 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.215329 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.215443 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.215603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.215719 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.connection = **** {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.215834 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.215948 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.216062 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.216175 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.216298 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.216413 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.216526 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.216526 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.216739 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.216853 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.216977 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.217092 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.217206 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.217321 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.217434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.217434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.217656 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.217656 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.217871 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.217986 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.218100 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.218214 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.218337 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.218453 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.218567 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.218682 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.218795 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.218909 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.219033 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.219179 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.219294 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.219408 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81520) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 13 12:56:21.219523 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.219523 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.219523 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 13 12:56:21.219523 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 13 12:56:21.244805 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81520) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.245030 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81520) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.245252 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81520) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.245685 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.245912 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.246135 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.254097 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81520) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.255038 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.281712 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81520) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.281962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81520) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.282615 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.282842 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.336179 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.337565 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81520) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.385340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81520) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.385564 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.385651 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81520) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.385796 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.540212 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81520) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.540393 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.540479 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81520) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.540613 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.540693 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81520) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.540768 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.540853 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81520) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.540929 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.541002 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81520) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.541075 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.541156 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81520) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.541239 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.564965 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81520) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.565191 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81520) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.565359 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.565441 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81520) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.565590 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.565674 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81520) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.565811 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81520) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.565890 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.566021 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81520) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.566104 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.566231 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81520) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.566315 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.566439 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.566651 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.577314 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81520) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.577622 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.590513 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81520) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.590617 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.590800 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81520) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.590921 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.591025 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81520) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.591179 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.591268 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81520) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.591399 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.591481 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81520) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.591612 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.591708 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81520) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.591825 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.591908 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81520) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.592034 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 13 12:56:21.609018 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.609223 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 13 12:56:21.609308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.609428 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 13 12:56:21.609561 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.609757 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 13 12:56:21.609847 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.609956 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 13 12:56:21.610063 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.610255 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.610336 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.610518 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.610607 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.610774 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.610856 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.611028 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.620215 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.620487 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.620581 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.620754 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.620835 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.621020 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.621102 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.621303 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.628443 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81520) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.628721 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.631318 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81520) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.631600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.631695 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.631878 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.631959 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.632155 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.632237 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.632433 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.632516 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.632684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.632764 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.632947 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.633028 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.633214 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.633297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.633477 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.633556 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.633741 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.633830 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.634000 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.634080 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.634261 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.656912 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.657192 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.658355 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81520) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.658637 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.658717 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.658920 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.663256 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81520) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.663536 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.663620 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.663811 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.663892 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.664088 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.670020 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81520) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.670303 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.670384 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.670638 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.675838 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81520) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.676132 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.676228 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.676416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.676499 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.676699 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.676781 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.676973 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.677056 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.677246 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.677329 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.677515 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.677606 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.677779 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.677860 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.678044 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.693831 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.694104 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.694191 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.694415 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.697919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81520) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.698249 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.701123 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.701414 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.701496 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.701767 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.706136 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81520) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.706563 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.709454 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81520) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.709799 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.712661 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.712956 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.713040 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.713241 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.713324 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.713519 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.713600 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.713800 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.713889 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.714083 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.714172 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.714363 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.729744 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81520) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.730027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.730108 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.730291 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.730372 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.730551 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.730632 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.730822 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.737354 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.737648 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.738862 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.739198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.740401 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.740686 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.740769 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.740987 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.743347 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81520) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.743627 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.743833 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.743916 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.744116 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.744199 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.744397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.744542 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.744677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.745072 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.756056 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81520) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.756335 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.756609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.756690 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.756883 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.756966 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.757188 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.757269 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.757466 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.757557 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.757773 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.757854 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.758124 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.758411 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.773952 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81520) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.774480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.776840 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81520) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.777388 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.779719 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.780275 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.781146 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.781828 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.782627 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81520) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.782906 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.783329 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.783473 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.783754 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.784165 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.789571 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.790525 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.790999 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81520) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.791945 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.793874 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.794263 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.794546 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.794780 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.794870 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.795201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.795489 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.795767 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.803308 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81520) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.803604 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.803879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.803959 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.804204 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.804482 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.810636 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81520) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.810915 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.811223 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.811349 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 13 12:56:21.811498 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.811630 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.811767 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.811908 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.812029 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.812182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.812293 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.812464 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.812569 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.812733 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.812827 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.813005 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.813087 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.813794 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.813975 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.814057 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.814488 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.814576 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.814752 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.814881 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.815017 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81520) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.815120 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.815376 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.815653 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 13 12:56:21.848950 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5607ddf419c0 pid: 81520 (default app) Oct 13 12:56:21.849755 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5607ddf419c0 pid: 81521 (default app) Oct 13 12:56:21.852161 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb69635-444e-4955-9f9e-c3dae68a2dc3 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:21.852447 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb69635-444e-4955-9f9e-c3dae68a2dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:21.852716 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb69635-444e-4955-9f9e-c3dae68a2dc3 None None] PATH_INFO: `/v3` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:21.856800 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1/1] 10.222.0.23 () {50 vars in 791 bytes} [Thu Oct 13 12:56:21 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:28.546009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fa14b-dba3-4d77-b960-cf4487a6a8f8 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:28.546526 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fa14b-dba3-4d77-b960-cf4487a6a8f8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:28.546526 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fa14b-dba3-4d77-b960-cf4487a6a8f8 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:28.551458 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1/2] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:56:28 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:56:28.556587 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e6204-64d7-48c6-9240-47e612a2085e None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:28.557151 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e6204-64d7-48c6-9240-47e612a2085e None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:28.557151 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e6204-64d7-48c6-9240-47e612a2085e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:28.626226 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: INFO dbcounter [None req-f58e6204-64d7-48c6-9240-47e612a2085e None None] Registered counter for database keystone Oct 13 12:56:28.644570 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG oslo_db.sqlalchemy.engines [None req-f58e6204-64d7-48c6-9240-47e612a2085e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81520) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 13 12:56:28.650568 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writer thread running {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 13 12:56:28.827324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG passlib.handlers.bcrypt [None req-f58e6204-64d7-48c6-9240-47e612a2085e None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81520) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 13 12:56:28.827708 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG passlib.handlers.bcrypt [None req-f58e6204-64d7-48c6-9240-47e612a2085e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81520) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 13 12:56:28.885214 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-f58e6204-64d7-48c6-9240-47e612a2085e None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:56:28.886799 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-f58e6204-64d7-48c6-9240-47e612a2085e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:56:29.012960 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 2/3] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:56:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 13 12:56:29.525349 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458fa14b-dba3-4d77-b960-cf4487a6a8f8 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:29.594386 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: INFO dbcounter [None req-458fa14b-dba3-4d77-b960-cf4487a6a8f8 None None] Registered counter for database keystone Oct 13 12:56:29.615731 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG oslo_db.sqlalchemy.engines [None req-458fa14b-dba3-4d77-b960-cf4487a6a8f8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81521) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 13 12:56:29.621740 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writer thread running {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 13 12:56:29.738525 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458fa14b-dba3-4d77-b960-cf4487a6a8f8 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:29.742773 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702ad7a-53d6-4cc0-8078-f783a9df7d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:29.743415 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702ad7a-53d6-4cc0-8078-f783a9df7d71 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:29.743696 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702ad7a-53d6-4cc0-8078-f783a9df7d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:29.743970 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702ad7a-53d6-4cc0-8078-f783a9df7d71 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:29.752634 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d702ad7a-53d6-4cc0-8078-f783a9df7d71 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:29.803110 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d702ad7a-53d6-4cc0-8078-f783a9df7d71 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 13 12:56:29.803459 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d702ad7a-53d6-4cc0-8078-f783a9df7d71 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:29.812686 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 2/4] 10.222.0.23 () {58 vars in 1324 bytes} [Thu Oct 13 12:56:29 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 13 12:56:29.818949 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e6204-64d7-48c6-9240-47e612a2085e None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:29.835773 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e6204-64d7-48c6-9240-47e612a2085e None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:29.840134 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc6f4b9-358e-4060-ae50-975681165c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:29.841219 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6f4b9-358e-4060-ae50-975681165c33 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:29.841500 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6f4b9-358e-4060-ae50-975681165c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:29.841770 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6f4b9-358e-4060-ae50-975681165c33 admin admin] PATH_INFO: `/v3/projects/9cb9d18ca96e4beea199d190db853658` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:29.843379 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc6f4b9-358e-4060-ae50-975681165c33 admin admin] RBAC: Authorizing `identity:get_project(project_id=9cb9d18ca96e4beea199d190db853658)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:29.896299 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: WARNING keystone.common.rbac_enforcer.enforcer [None req-adc6f4b9-358e-4060-ae50-975681165c33 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 13 12:56:29.896651 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc6f4b9-358e-4060-ae50-975681165c33 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:29.901105 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 3/5] 10.222.0.23 () {56 vars in 1207 bytes} [Thu Oct 13 12:56:29 2022] GET /identity/v3/projects/9cb9d18ca96e4beea199d190db853658 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:31.994727 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3407a34-24c5-4c0b-8a15-a7aa1b9be922 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:31.995247 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3407a34-24c5-4c0b-8a15-a7aa1b9be922 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:31.995339 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3407a34-24c5-4c0b-8a15-a7aa1b9be922 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:31.999333 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 3/6] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:56:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:56:32.005369 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e75e78-d0a3-45cb-8dd7-70a7642f4e28 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:32.005776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e75e78-d0a3-45cb-8dd7-70a7642f4e28 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:32.005913 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e75e78-d0a3-45cb-8dd7-70a7642f4e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:32.025606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-b5e75e78-d0a3-45cb-8dd7-70a7642f4e28 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:56:32.027120 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-b5e75e78-d0a3-45cb-8dd7-70a7642f4e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:56:32.034689 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 4/7] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:56:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 13 12:56:32.301617 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3407a34-24c5-4c0b-8a15-a7aa1b9be922 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:32.313550 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3407a34-24c5-4c0b-8a15-a7aa1b9be922 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:32.317984 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d5232-d4f9-4d0d-b26f-f08860c8148e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:32.318576 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d5232-d4f9-4d0d-b26f-f08860c8148e admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:32.318857 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d5232-d4f9-4d0d-b26f-f08860c8148e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:32.319157 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d5232-d4f9-4d0d-b26f-f08860c8148e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:32.326594 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d5232-d4f9-4d0d-b26f-f08860c8148e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:32.328409 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d5232-d4f9-4d0d-b26f-f08860c8148e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:32.336957 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 4/8] 10.222.0.23 () {58 vars in 1324 bytes} [Thu Oct 13 12:56:32 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 13 12:56:32.342037 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e75e78-d0a3-45cb-8dd7-70a7642f4e28 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:32.351432 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e75e78-d0a3-45cb-8dd7-70a7642f4e28 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:32.355700 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a77ba-fa5b-4de8-9a98-b878bd7c4aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:32.356336 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a77ba-fa5b-4de8-9a98-b878bd7c4aa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:32.356587 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a77ba-fa5b-4de8-9a98-b878bd7c4aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:32.356854 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a77ba-fa5b-4de8-9a98-b878bd7c4aa0 admin admin] PATH_INFO: `/v3/users/22269fc49ecd49a7b97d86f5937fc2dc` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:32.358137 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133a77ba-fa5b-4de8-9a98-b878bd7c4aa0 admin admin] RBAC: Authorizing `identity:get_user(user_id=22269fc49ecd49a7b97d86f5937fc2dc)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:32.360595 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133a77ba-fa5b-4de8-9a98-b878bd7c4aa0 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:32.362074 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 5/9] 10.222.0.23 () {56 vars in 1198 bytes} [Thu Oct 13 12:56:32 2022] GET /identity/v3/users/22269fc49ecd49a7b97d86f5937fc2dc => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:34.525311 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5af8f8-ccc0-4356-bb2d-e32a9c5724b5 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:34.525847 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5af8f8-ccc0-4356-bb2d-e32a9c5724b5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:34.525847 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5af8f8-ccc0-4356-bb2d-e32a9c5724b5 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:34.527194 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 5/10] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:56:34.532241 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3226e64-19f4-4978-b645-63bedd010f6f None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:34.532621 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3226e64-19f4-4978-b645-63bedd010f6f None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:34.532918 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3226e64-19f4-4978-b645-63bedd010f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:34.550733 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-a3226e64-19f4-4978-b645-63bedd010f6f None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:56:34.551965 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-a3226e64-19f4-4978-b645-63bedd010f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:56:34.574519 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a568f0-eb5b-4e44-adc5-f6e5d263d9dd None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:34.574826 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a568f0-eb5b-4e44-adc5-f6e5d263d9dd None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:34.575138 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a568f0-eb5b-4e44-adc5-f6e5d263d9dd None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:34.576450 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 6/11] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 13 12:56:34.582161 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d470794-ddfc-4c2b-b0f5-7c85e57eb83b None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:34.582443 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d470794-ddfc-4c2b-b0f5-7c85e57eb83b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:34.582713 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d470794-ddfc-4c2b-b0f5-7c85e57eb83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:34.624159 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG passlib.handlers.bcrypt [None req-1d470794-ddfc-4c2b-b0f5-7c85e57eb83b None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81521) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 13 12:56:34.624550 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG passlib.handlers.bcrypt [None req-1d470794-ddfc-4c2b-b0f5-7c85e57eb83b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81521) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 13 12:56:34.633789 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 6/12] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:56:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 13 12:56:34.673809 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-1d470794-ddfc-4c2b-b0f5-7c85e57eb83b None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:56:34.675423 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1d470794-ddfc-4c2b-b0f5-7c85e57eb83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:56:34.773011 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 7/13] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:56:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 13 12:56:34.908141 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3226e64-19f4-4978-b645-63bedd010f6f None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:34.933905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:35.012741 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3226e64-19f4-4978-b645-63bedd010f6f None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:35.035880 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d470794-ddfc-4c2b-b0f5-7c85e57eb83b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:35.045044 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c1957-0f9f-41de-9f2c-a7df1effdbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:35.045681 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c1957-0f9f-41de-9f2c-a7df1effdbe9 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:35.045978 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c1957-0f9f-41de-9f2c-a7df1effdbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:35.046251 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c1957-0f9f-41de-9f2c-a7df1effdbe9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:35.057988 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c1957-0f9f-41de-9f2c-a7df1effdbe9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:56:35.067655 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-eb9c1957-0f9f-41de-9f2c-a7df1effdbe9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:56:35.067655 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:56:35.067655 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:56:35.067655 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:56:35.067655 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:56:35.067655 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:56:35.067655 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:56:35.067655 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:56:35.067655 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:56:35.067655 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:56:35.067655 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:56:35.067655 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:56:35.067655 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:56:35.067655 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:56:35.067655 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:56:35.067655 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:56:35.068553 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:56:35.068553 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:56:35.068553 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:56:35.068553 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:56:35.068553 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:56:35.068553 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:56:35.068553 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:56:35.068553 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:56:35.068553 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:56:35.068553 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:56:35.068553 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:56:35.068553 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:56:35.068553 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:56:35.068553 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:56:35.068553 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:56:35.068553 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:56:35.069438 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:56:35.069438 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:56:35.069438 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:56:35.069438 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:56:35.069438 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:56:35.069438 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:56:35.069438 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:56:35.069438 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:56:35.069438 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:56:35.069438 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:56:35.069438 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:56:35.069438 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:56:35.069438 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 7/14] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 13 12:56:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:56:35.074470 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c1957-0f9f-41de-9f2c-a7df1effdbe9 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:35.135591 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d470794-ddfc-4c2b-b0f5-7c85e57eb83b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:35.137803 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:35.153909 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:35.172608 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c1957-0f9f-41de-9f2c-a7df1effdbe9 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:35.173550 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254c086-3d65-4abf-bed2-ba369cfad8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:35.174411 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254c086-3d65-4abf-bed2-ba369cfad8e6 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:35.174693 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254c086-3d65-4abf-bed2-ba369cfad8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:35.174967 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254c086-3d65-4abf-bed2-ba369cfad8e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:35.175962 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254c086-3d65-4abf-bed2-ba369cfad8e6 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:35.177378 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254c086-3d65-4abf-bed2-ba369cfad8e6 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:35.187988 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.sql.core [None req-b254c086-3d65-4abf-bed2-ba369cfad8e6 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 13 12:56:35.187988 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 13 12:56:35.187988 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [parameters: {'id': '79b73d19deb349b3be8b9878d1908912', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 13 12:56:35.187988 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81521) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 13 12:56:35.205597 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f65028-183f-4090-b231-14fc1e431fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:35.206209 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f65028-183f-4090-b231-14fc1e431fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:35.206491 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f65028-183f-4090-b231-14fc1e431fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:35.206579 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-b254c086-3d65-4abf-bed2-ba369cfad8e6 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 13 12:56:35.206579 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:56:35.206579 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 13 12:56:35.206579 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 13 12:56:35.206579 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 13 12:56:35.206579 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 13 12:56:35.206579 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 13 12:56:35.206579 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result = self._query(query) Oct 13 12:56:35.206579 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 13 12:56:35.206579 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application conn.query(q) Oct 13 12:56:35.206579 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 13 12:56:35.206579 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 13 12:56:35.206579 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 13 12:56:35.206579 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result.read() Oct 13 12:56:35.207409 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 13 12:56:35.207409 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 13 12:56:35.207409 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 13 12:56:35.207409 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 13 12:56:35.207409 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 13 12:56:35.207409 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 13 12:56:35.207409 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 13 12:56:35.207409 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 13 12:56:35.207409 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 13 12:56:35.207409 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:56:35.207409 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 13 12:56:35.207409 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:56:35.207409 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:56:35.207409 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 13 12:56:35.207409 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 13 12:56:35.207409 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 13 12:56:35.207409 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 13 12:56:35.207409 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 13 12:56:35.208302 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application next(self.gen) Oct 13 12:56:35.208302 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 13 12:56:35.208302 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application yield resource Oct 13 12:56:35.208302 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 13 12:56:35.208302 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application next(self.gen) Oct 13 12:56:35.208302 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 13 12:56:35.208302 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.session.rollback() Oct 13 12:56:35.208302 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 13 12:56:35.208302 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.force_reraise() Oct 13 12:56:35.208302 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 13 12:56:35.208302 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise self.value Oct 13 12:56:35.208302 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 13 12:56:35.208302 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 13 12:56:35.208302 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 13 12:56:35.208302 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application session.commit() Oct 13 12:56:35.208302 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 13 12:56:35.208302 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 13 12:56:35.209119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 13 12:56:35.209119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._prepare_impl() Oct 13 12:56:35.209119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 13 12:56:35.209119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.session.flush() Oct 13 12:56:35.209119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 13 12:56:35.209119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._flush(objects) Oct 13 12:56:35.209119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 13 12:56:35.209119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 13 12:56:35.209119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 13 12:56:35.209119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application compat.raise_( Oct 13 12:56:35.209119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 13 12:56:35.209119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception Oct 13 12:56:35.209119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 13 12:56:35.209119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application flush_context.execute() Oct 13 12:56:35.209119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 13 12:56:35.209119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rec.execute(self) Oct 13 12:56:35.209119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 13 12:56:35.209962 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 13 12:56:35.209962 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 13 12:56:35.209962 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 13 12:56:35.209962 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 13 12:56:35.209962 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 13 12:56:35.209962 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 13 12:56:35.209962 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 13 12:56:35.209962 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 13 12:56:35.209962 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 13 12:56:35.209962 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 13 12:56:35.209962 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 13 12:56:35.209962 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 13 12:56:35.209962 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 13 12:56:35.209962 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 13 12:56:35.209962 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 13 12:56:35.210735 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 13 12:56:35.210735 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception Oct 13 12:56:35.210735 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 13 12:56:35.210735 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 13 12:56:35.210735 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 13 12:56:35.210735 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 13 12:56:35.210735 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 13 12:56:35.210735 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result = self._query(query) Oct 13 12:56:35.210735 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 13 12:56:35.210735 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application conn.query(q) Oct 13 12:56:35.210735 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 13 12:56:35.210735 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 13 12:56:35.210735 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 13 12:56:35.210735 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result.read() Oct 13 12:56:35.210735 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 13 12:56:35.210735 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 13 12:56:35.210735 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 13 12:56:35.211596 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 13 12:56:35.211596 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 13 12:56:35.211596 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 13 12:56:35.211596 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 13 12:56:35.211596 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 13 12:56:35.211596 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 13 12:56:35.211596 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 13 12:56:35.211596 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [parameters: {'id': '79b73d19deb349b3be8b9878d1908912', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 13 12:56:35.211596 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 13 12:56:35.211596 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:56:35.211596 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 12:56:35.211596 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:56:35.211596 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:56:35.211596 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 13 12:56:35.211596 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 13 12:56:35.212292 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 13 12:56:35.212292 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 13 12:56:35.212292 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 13 12:56:35.212292 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:56:35.212292 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 12:56:35.212292 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:56:35.212292 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:56:35.212292 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:56:35.212292 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:56:35.212292 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:56:35.212292 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:56:35.212292 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:56:35.212292 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:56:35.212292 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:56:35.212292 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:56:35.212292 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:56:35.212292 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:56:35.213189 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 13 12:56:35.213189 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 13 12:56:35.213189 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:56:35.213189 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:56:35.213189 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 13 12:56:35.213189 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 13 12:56:35.213189 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:56:35.213189 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:56:35.213189 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 13 12:56:35.213189 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 13 12:56:35.213189 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 13 12:56:35.213189 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:56:35.213781 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f65028-183f-4090-b231-14fc1e431fd2 None admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:35.213781 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f65028-183f-4090-b231-14fc1e431fd2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:56:35.213781 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f65028-183f-4090-b231-14fc1e431fd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:35.213967 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 8/15] 10.222.0.23 () {60 vars in 1137 bytes} [Thu Oct 13 12:56:35 2022] POST /identity/v3/domains => generated 190 bytes in 172 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:56:35.214035 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f65028-183f-4090-b231-14fc1e431fd2 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:35.214098 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254c086-3d65-4abf-bed2-ba369cfad8e6 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:35.222782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 8/16] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:56:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:56:35.228421 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f65028-183f-4090-b231-14fc1e431fd2 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:35.313325 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254c086-3d65-4abf-bed2-ba369cfad8e6 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:35.327998 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f65028-183f-4090-b231-14fc1e431fd2 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:35.346949 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08754c-77d7-4d0a-a801-58a17f734f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:35.347854 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08754c-77d7-4d0a-a801-58a17f734f5d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:35.348131 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08754c-77d7-4d0a-a801-58a17f734f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:35.348404 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08754c-77d7-4d0a-a801-58a17f734f5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:35.361175 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88bb19a-8da3-4bae-9e47-4f5181960ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:35.361788 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88bb19a-8da3-4bae-9e47-4f5181960ade None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:35.362063 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88bb19a-8da3-4bae-9e47-4f5181960ade None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:35.362331 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88bb19a-8da3-4bae-9e47-4f5181960ade None admin] PATH_INFO: `/v3/users/22269fc49ecd49a7b97d86f5937fc2dc` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:35.363485 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88bb19a-8da3-4bae-9e47-4f5181960ade None admin] RBAC: Authorizing `identity:get_user(user_id=22269fc49ecd49a7b97d86f5937fc2dc)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:35.364983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88bb19a-8da3-4bae-9e47-4f5181960ade None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:35.366504 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 9/17] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:56:35 2022] GET /identity/v3/users/22269fc49ecd49a7b97d86f5937fc2dc => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:56:35.366600 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a08754c-77d7-4d0a-a801-58a17f734f5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:35.368091 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a08754c-77d7-4d0a-a801-58a17f734f5d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:35.372001 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88bb19a-8da3-4bae-9e47-4f5181960ade None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:35.383866 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-3a08754c-77d7-4d0a-a801-58a17f734f5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:56:35.383866 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:56:35.383866 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:56:35.383866 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:56:35.383866 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:56:35.383866 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:56:35.383866 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:56:35.383866 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:56:35.383866 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:56:35.383866 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:56:35.383866 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:56:35.383866 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:56:35.383866 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:56:35.383866 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:56:35.383866 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:56:35.383866 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:56:35.384911 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:56:35.384911 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:56:35.384911 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:56:35.384911 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:56:35.384911 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:56:35.384911 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:56:35.384911 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:56:35.384911 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:56:35.384911 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:56:35.384911 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:56:35.384911 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:56:35.384911 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:56:35.384911 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:56:35.384911 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:56:35.384911 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:56:35.384911 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:56:35.384911 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:56:35.385914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:56:35.385914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:56:35.385914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:56:35.385914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 9/18] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:56:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:56:35.388135 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:35.390913 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08754c-77d7-4d0a-a801-58a17f734f5d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:35.412042 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:35.473812 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88bb19a-8da3-4bae-9e47-4f5181960ade None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:35.492492 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08754c-77d7-4d0a-a801-58a17f734f5d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:35.506217 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f700c8d-d2d2-4a31-886b-07a52e7fabca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:35.506849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f700c8d-d2d2-4a31-886b-07a52e7fabca None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:35.507158 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f700c8d-d2d2-4a31-886b-07a52e7fabca None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:35.507435 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f700c8d-d2d2-4a31-886b-07a52e7fabca None admin] PATH_INFO: `/v3/domains/default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:35.508552 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f700c8d-d2d2-4a31-886b-07a52e7fabca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:35.510019 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f700c8d-d2d2-4a31-886b-07a52e7fabca None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:35.511595 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 10/19] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:56:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:56:35.517674 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f700c8d-d2d2-4a31-886b-07a52e7fabca None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:35.525138 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6000fc-ac23-4abc-af5a-9a46ae467fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:35.525740 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6000fc-ac23-4abc-af5a-9a46ae467fb3 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:35.526276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6000fc-ac23-4abc-af5a-9a46ae467fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:35.526407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6000fc-ac23-4abc-af5a-9a46ae467fb3 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:35.527045 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6000fc-ac23-4abc-af5a-9a46ae467fb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:56:35.527371 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6000fc-ac23-4abc-af5a-9a46ae467fb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:35.528737 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6000fc-ac23-4abc-af5a-9a46ae467fb3 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:35.546498 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 10/20] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:35.601794 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:35.621946 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f700c8d-d2d2-4a31-886b-07a52e7fabca None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:35.657218 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ea04f-4f39-450c-abca-bac2ec023a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:35.657900 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ea04f-4f39-450c-abca-bac2ec023a37 None admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:35.658181 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ea04f-4f39-450c-abca-bac2ec023a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:35.658456 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ea04f-4f39-450c-abca-bac2ec023a37 None admin] PATH_INFO: `/v3/domains/default/users/22269fc49ecd49a7b97d86f5937fc2dc/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:35.660486 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ea04f-4f39-450c-abca-bac2ec023a37 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=22269fc49ecd49a7b97d86f5937fc2dc, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:35.661952 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ea04f-4f39-450c-abca-bac2ec023a37 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:35.776942 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 11/21] 10.222.0.23 () {58 vars in 1361 bytes} [Thu Oct 13 12:56:35 2022] PUT /identity/v3/domains/default/users/22269fc49ecd49a7b97d86f5937fc2dc/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 12:56:38.294092 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f3d84-5181-4e9d-a3cb-6113541514d5 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:38.294578 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f3d84-5181-4e9d-a3cb-6113541514d5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:38.294578 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f3d84-5181-4e9d-a3cb-6113541514d5 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:38.296213 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 11/22] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:56:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:56:38.302502 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925ac9c-b54d-4775-a20d-4540accbd544 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:38.303385 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925ac9c-b54d-4775-a20d-4540accbd544 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:38.303887 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925ac9c-b54d-4775-a20d-4540accbd544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:38.332231 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-d925ac9c-b54d-4775-a20d-4540accbd544 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:56:38.334106 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-d925ac9c-b54d-4775-a20d-4540accbd544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:56:38.420947 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 12/23] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:56:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:56:38.469039 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b93857-00ad-48be-bbab-e0016eb6615b None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:38.469401 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b93857-00ad-48be-bbab-e0016eb6615b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:38.469891 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b93857-00ad-48be-bbab-e0016eb6615b None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:38.471587 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 12/24] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:56:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:56:38.476919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b53e6-4dca-4bc4-8150-42009bc5bcc2 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:38.477209 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b53e6-4dca-4bc4-8150-42009bc5bcc2 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:38.477492 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b53e6-4dca-4bc4-8150-42009bc5bcc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:38.482811 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9d278-902b-42d2-bc98-9670d1274732 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:38.483387 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9d278-902b-42d2-bc98-9670d1274732 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:38.484436 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9d278-902b-42d2-bc98-9670d1274732 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:38.486045 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 13/25] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:56:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 13 12:56:38.491762 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a56865-dd70-44cb-8ea6-06a332ffadbb None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:38.492279 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a56865-dd70-44cb-8ea6-06a332ffadbb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:38.492749 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a56865-dd70-44cb-8ea6-06a332ffadbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:38.525726 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-d85b53e6-4dca-4bc4-8150-42009bc5bcc2 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:56:38.526966 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-d85b53e6-4dca-4bc4-8150-42009bc5bcc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:56:38.534196 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-01a56865-dd70-44cb-8ea6-06a332ffadbb None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:56:38.535441 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-01a56865-dd70-44cb-8ea6-06a332ffadbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:56:38.644437 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 14/26] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:56:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 13 12:56:38.680400 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 13/27] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:56:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 13 12:56:38.681087 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:38.711577 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67410f2-b7c6-4379-b461-cae4fcc08e0f None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:38.712177 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67410f2-b7c6-4379-b461-cae4fcc08e0f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:38.712650 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67410f2-b7c6-4379-b461-cae4fcc08e0f None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:38.714178 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 15/28] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:56:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:56:38.719359 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc312d0b-a4a2-4c0e-aee6-7d86dbb3f3e0 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:38.719892 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc312d0b-a4a2-4c0e-aee6-7d86dbb3f3e0 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:38.720384 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc312d0b-a4a2-4c0e-aee6-7d86dbb3f3e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:38.745111 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-dc312d0b-a4a2-4c0e-aee6-7d86dbb3f3e0 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:56:38.746620 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-dc312d0b-a4a2-4c0e-aee6-7d86dbb3f3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:56:38.837345 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 14/29] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:56:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:56:38.918939 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67410f2-b7c6-4379-b461-cae4fcc08e0f None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:38.937473 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:39.050012 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67410f2-b7c6-4379-b461-cae4fcc08e0f None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:39.060128 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc312d0b-a4a2-4c0e-aee6-7d86dbb3f3e0 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:39.122334 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb67d5a-df58-473d-a7c4-77c9d2778813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:39.123388 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb67d5a-df58-473d-a7c4-77c9d2778813 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:39.123870 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb67d5a-df58-473d-a7c4-77c9d2778813 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:39.124890 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb67d5a-df58-473d-a7c4-77c9d2778813 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:39.125763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb67d5a-df58-473d-a7c4-77c9d2778813 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:39.127455 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb67d5a-df58-473d-a7c4-77c9d2778813 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:39.251506 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc312d0b-a4a2-4c0e-aee6-7d86dbb3f3e0 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:39.295991 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:39.297870 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 16/30] 10.222.0.23 () {60 vars in 1131 bytes} [Thu Oct 13 12:56:38 2022] POST /identity/v3/roles => generated 219 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:56:39.299665 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb67d5a-df58-473d-a7c4-77c9d2778813 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:39.316161 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed531a-b64b-4de7-8009-89d693cfd105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:39.317432 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed531a-b64b-4de7-8009-89d693cfd105 None admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:39.317874 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed531a-b64b-4de7-8009-89d693cfd105 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:39.318287 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed531a-b64b-4de7-8009-89d693cfd105 None admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:39.319184 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ed531a-b64b-4de7-8009-89d693cfd105 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:39.321004 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ed531a-b64b-4de7-8009-89d693cfd105 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:39.354265 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 15/31] 10.222.0.23 () {60 vars in 1131 bytes} [Thu Oct 13 12:56:39 2022] POST /identity/v3/roles => generated 225 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 12:56:39.356328 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed531a-b64b-4de7-8009-89d693cfd105 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:39.457692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:39.526557 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb67d5a-df58-473d-a7c4-77c9d2778813 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:39.570833 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed531a-b64b-4de7-8009-89d693cfd105 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:39.584367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e4297-86dc-47b1-905d-37e80aa3b2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:39.585045 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e4297-86dc-47b1-905d-37e80aa3b2c4 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:39.585489 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e4297-86dc-47b1-905d-37e80aa3b2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:39.585921 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e4297-86dc-47b1-905d-37e80aa3b2c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:39.602432 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5e4297-86dc-47b1-905d-37e80aa3b2c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:56:39.610136 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-ed5e4297-86dc-47b1-905d-37e80aa3b2c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:56:39.610136 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:56:39.610136 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:56:39.610136 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:56:39.610136 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:56:39.610136 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:56:39.610136 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:56:39.610136 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:56:39.610136 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:56:39.610136 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:56:39.610136 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:56:39.610136 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:56:39.610136 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:56:39.610136 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:56:39.610136 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:56:39.610136 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:56:39.611252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:56:39.611252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:56:39.611252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:56:39.611252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:56:39.611252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:56:39.611252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:56:39.611252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:56:39.611252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:56:39.611252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:56:39.611252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:56:39.611252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:56:39.611252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:56:39.611252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:56:39.611252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:56:39.611252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:56:39.611252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:56:39.612051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:56:39.612051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:56:39.612051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:56:39.612051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:56:39.612051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:56:39.612051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:56:39.612051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:56:39.612051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:56:39.612051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:56:39.612051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:56:39.612051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:56:39.612051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:56:39.614270 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 17/32] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 13 12:56:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:56:39.616040 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f7e15-eb10-41e3-9747-c5d1a198db8d None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:39.616564 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f7e15-eb10-41e3-9747-c5d1a198db8d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:39.617045 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f7e15-eb10-41e3-9747-c5d1a198db8d None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:39.618567 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 18/33] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:56:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:56:39.623728 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be28f40-2a35-461a-8509-65c261efb48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:39.624079 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f7e15-eb10-41e3-9747-c5d1a198db8d None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:39.624801 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be28f40-2a35-461a-8509-65c261efb48d None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:39.625226 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be28f40-2a35-461a-8509-65c261efb48d None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:39.625640 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be28f40-2a35-461a-8509-65c261efb48d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:39.640767 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be28f40-2a35-461a-8509-65c261efb48d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:39.642280 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be28f40-2a35-461a-8509-65c261efb48d None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:39.660501 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-9be28f40-2a35-461a-8509-65c261efb48d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:56:39.660501 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:56:39.660501 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:56:39.660501 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:56:39.660501 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:56:39.660501 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:56:39.660501 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:56:39.660501 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:56:39.660501 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:56:39.660501 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:56:39.660501 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:56:39.660501 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:56:39.660501 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:56:39.660501 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:56:39.660501 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:56:39.660501 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:56:39.661582 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:56:39.661582 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:56:39.661582 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:56:39.661582 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:56:39.661582 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:56:39.661582 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:56:39.661582 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:56:39.661582 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:56:39.661582 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:56:39.661582 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:56:39.661582 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:56:39.661582 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:56:39.661582 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:56:39.661582 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:56:39.661582 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:56:39.661582 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:56:39.661582 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:56:39.662439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:56:39.662439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:56:39.662439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:56:39.666305 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 16/34] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:56:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:56:39.666305 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd3224-553b-4ff9-88da-8bfe0a37b791 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:39.666305 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd3224-553b-4ff9-88da-8bfe0a37b791 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:39.666305 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd3224-553b-4ff9-88da-8bfe0a37b791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:39.689637 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-68cd3224-553b-4ff9-88da-8bfe0a37b791 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:56:39.690809 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-68cd3224-553b-4ff9-88da-8bfe0a37b791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:56:39.730594 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:39.760252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f7e15-eb10-41e3-9747-c5d1a198db8d None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:39.771153 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 17/35] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:56:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:56:39.772374 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd3224-553b-4ff9-88da-8bfe0a37b791 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:39.799419 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:39.806916 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2919bb-2ccb-4c1b-8c1e-38b912682b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:39.807866 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2919bb-2ccb-4c1b-8c1e-38b912682b62 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:39.808407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2919bb-2ccb-4c1b-8c1e-38b912682b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:39.808924 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2919bb-2ccb-4c1b-8c1e-38b912682b62 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:39.809796 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2919bb-2ccb-4c1b-8c1e-38b912682b62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:56:39.810286 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2919bb-2ccb-4c1b-8c1e-38b912682b62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:39.812743 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2919bb-2ccb-4c1b-8c1e-38b912682b62 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:39.826058 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 19/36] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:56:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:39.833484 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2919bb-2ccb-4c1b-8c1e-38b912682b62 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:39.903896 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd3224-553b-4ff9-88da-8bfe0a37b791 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:39.952273 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe8f8e-3e1f-4cd6-8ec2-c05bb1998b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:39.953453 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe8f8e-3e1f-4cd6-8ec2-c05bb1998b09 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:39.953735 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe8f8e-3e1f-4cd6-8ec2-c05bb1998b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:39.954009 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe8f8e-3e1f-4cd6-8ec2-c05bb1998b09 None admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:39.954906 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe8f8e-3e1f-4cd6-8ec2-c05bb1998b09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:56:39.955008 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe8f8e-3e1f-4cd6-8ec2-c05bb1998b09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:39.956395 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe8f8e-3e1f-4cd6-8ec2-c05bb1998b09 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:39.971786 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2919bb-2ccb-4c1b-8c1e-38b912682b62 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:40.012739 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325434a-ba85-4425-9773-936402ee2f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:40.013756 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325434a-ba85-4425-9773-936402ee2f60 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:40.014264 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325434a-ba85-4425-9773-936402ee2f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:40.014758 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325434a-ba85-4425-9773-936402ee2f60 None admin] PATH_INFO: `/v3/users/22269fc49ecd49a7b97d86f5937fc2dc` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:40.016879 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e325434a-ba85-4425-9773-936402ee2f60 None admin] RBAC: Authorizing `identity:get_user(user_id=22269fc49ecd49a7b97d86f5937fc2dc)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:40.018738 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e325434a-ba85-4425-9773-936402ee2f60 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:40.025293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 20/37] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:56:39 2022] GET /identity/v3/users/22269fc49ecd49a7b97d86f5937fc2dc => generated 247 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:40.027091 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325434a-ba85-4425-9773-936402ee2f60 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:40.081031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 18/38] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:56:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:40.082218 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1db9f7-93f3-429f-b5b2-dc23f3b0f630 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:40.082503 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1db9f7-93f3-429f-b5b2-dc23f3b0f630 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:40.082777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1db9f7-93f3-429f-b5b2-dc23f3b0f630 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:40.084005 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 19/39] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:56:40.087814 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1db9f7-93f3-429f-b5b2-dc23f3b0f630 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:40.095846 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:40.119873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:40.137416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325434a-ba85-4425-9773-936402ee2f60 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:40.169541 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dfbd29-7c54-4a60-a380-26297f431dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:40.170175 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dfbd29-7c54-4a60-a380-26297f431dcb None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:40.170457 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dfbd29-7c54-4a60-a380-26297f431dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:40.170738 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dfbd29-7c54-4a60-a380-26297f431dcb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:40.172488 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dfbd29-7c54-4a60-a380-26297f431dcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:40.173929 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dfbd29-7c54-4a60-a380-26297f431dcb None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:40.175519 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 21/40] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:56:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:56:40.176287 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dfbd29-7c54-4a60-a380-26297f431dcb None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:40.193055 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1db9f7-93f3-429f-b5b2-dc23f3b0f630 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:40.226329 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf407f-7161-4615-933d-0afa041a4478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:40.226938 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf407f-7161-4615-933d-0afa041a4478 None admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:40.227250 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf407f-7161-4615-933d-0afa041a4478 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:40.227526 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf407f-7161-4615-933d-0afa041a4478 None admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:40.228190 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cf407f-7161-4615-933d-0afa041a4478 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:40.229581 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cf407f-7161-4615-933d-0afa041a4478 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:40.243887 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 20/41] 10.222.0.23 () {60 vars in 1131 bytes} [Thu Oct 13 12:56:40 2022] POST /identity/v3/roles => generated 223 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:56:40.245209 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ded329-2927-4ec9-9f69-937df746445e None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:40.245496 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ded329-2927-4ec9-9f69-937df746445e None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:40.245771 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ded329-2927-4ec9-9f69-937df746445e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:40.264676 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-72ded329-2927-4ec9-9f69-937df746445e None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:56:40.265893 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-72ded329-2927-4ec9-9f69-937df746445e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:56:40.289828 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dfbd29-7c54-4a60-a380-26297f431dcb None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:40.321358 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:40.331947 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2fbcd-88b1-4c45-8a75-224894718a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:40.332486 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2fbcd-88b1-4c45-8a75-224894718a60 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:40.332771 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2fbcd-88b1-4c45-8a75-224894718a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:40.333045 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2fbcd-88b1-4c45-8a75-224894718a60 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:40.333714 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b2fbcd-88b1-4c45-8a75-224894718a60 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:40.335219 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b2fbcd-88b1-4c45-8a75-224894718a60 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:40.341025 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 21/42] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:56:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:56:40.341703 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ded329-2927-4ec9-9f69-937df746445e None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:40.373426 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:40.382628 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 22/43] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Oct 13 12:56:40 2022] POST /identity/v3/projects => generated 301 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:56:40.471329 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ded329-2927-4ec9-9f69-937df746445e None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:40.510016 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413e04b-74ad-4885-ae96-e67c06ab1c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:40.510654 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413e04b-74ad-4885-ae96-e67c06ab1c7d None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:40.510939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413e04b-74ad-4885-ae96-e67c06ab1c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:40.511244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413e04b-74ad-4885-ae96-e67c06ab1c7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:40.512010 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6413e04b-74ad-4885-ae96-e67c06ab1c7d None admin] RBAC: Adding query filter params (role.id=70af94fc6a8149f2a38f74b843e1a9f0, scope.domain.id=default, user.id=22269fc49ecd49a7b97d86f5937fc2dc) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:56:40.512299 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6413e04b-74ad-4885-ae96-e67c06ab1c7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:40.513828 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6413e04b-74ad-4885-ae96-e67c06ab1c7d None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:40.522258 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 22/44] 10.222.0.23 () {56 vars in 1322 bytes} [Thu Oct 13 12:56:40 2022] GET /identity/v3/role_assignments?user.id=22269fc49ecd49a7b97d86f5937fc2dc&scope.domain.id=default&role.id=70af94fc6a8149f2a38f74b843e1a9f0 => generated 530 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:40.615233 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2fbcd-88b1-4c45-8a75-224894718a60 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:40.718655 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2fbcd-88b1-4c45-8a75-224894718a60 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:40.752050 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dcadeb-2161-4904-b585-957b96f7cb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:40.752682 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcadeb-2161-4904-b585-957b96f7cb26 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:40.752967 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcadeb-2161-4904-b585-957b96f7cb26 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:40.753242 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcadeb-2161-4904-b585-957b96f7cb26 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:40.754337 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dcadeb-2161-4904-b585-957b96f7cb26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:40.755803 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dcadeb-2161-4904-b585-957b96f7cb26 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:40.757365 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 23/45] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:56:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:40.762974 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413e04b-74ad-4885-ae96-e67c06ab1c7d None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:40.847256 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:40.861732 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413e04b-74ad-4885-ae96-e67c06ab1c7d None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:40.899914 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3ff3b-0c94-4f21-8f7d-406db584ec14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:40.900462 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3ff3b-0c94-4f21-8f7d-406db584ec14 None admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:40.900750 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3ff3b-0c94-4f21-8f7d-406db584ec14 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:40.901032 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3ff3b-0c94-4f21-8f7d-406db584ec14 None admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:40.901688 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee3ff3b-0c94-4f21-8f7d-406db584ec14 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:40.903184 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee3ff3b-0c94-4f21-8f7d-406db584ec14 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:40.931273 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 23/46] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Oct 13 12:56:40 2022] POST /identity/v3/projects => generated 312 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:56:43.073000 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40967428-c041-44fd-a2b9-985599b3ea1e None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:43.073504 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40967428-c041-44fd-a2b9-985599b3ea1e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:43.073584 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40967428-c041-44fd-a2b9-985599b3ea1e None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:43.074860 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 24/47] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:56:43.080592 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcba251-9897-45a1-bd9f-b9db7ba004fd None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:43.081003 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcba251-9897-45a1-bd9f-b9db7ba004fd None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:43.081143 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcba251-9897-45a1-bd9f-b9db7ba004fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:43.099507 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-1fcba251-9897-45a1-bd9f-b9db7ba004fd None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:56:43.100671 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-1fcba251-9897-45a1-bd9f-b9db7ba004fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:56:43.164316 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 24/48] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:56:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:56:43.428041 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40967428-c041-44fd-a2b9-985599b3ea1e None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:43.476832 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:43.526929 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40967428-c041-44fd-a2b9-985599b3ea1e None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:43.559404 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e84f6-0273-4d81-927d-8ed386d04eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:43.560025 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e84f6-0273-4d81-927d-8ed386d04eee None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:43.560308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e84f6-0273-4d81-927d-8ed386d04eee None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:43.560581 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e84f6-0273-4d81-927d-8ed386d04eee None admin] PATH_INFO: `/v3/domains/default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:43.561684 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45e84f6-0273-4d81-927d-8ed386d04eee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:43.563919 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45e84f6-0273-4d81-927d-8ed386d04eee None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:43.565463 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 25/49] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:56:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:43.570990 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcba251-9897-45a1-bd9f-b9db7ba004fd None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:43.633999 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:43.666824 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcba251-9897-45a1-bd9f-b9db7ba004fd None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:43.699462 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb1961-f354-41c4-a4e8-3c817fb00dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:43.700011 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb1961-f354-41c4-a4e8-3c817fb00dd0 None admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:43.700295 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb1961-f354-41c4-a4e8-3c817fb00dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:43.700570 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb1961-f354-41c4-a4e8-3c817fb00dd0 None admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:43.701216 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb1961-f354-41c4-a4e8-3c817fb00dd0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:43.702645 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecb1961-f354-41c4-a4e8-3c817fb00dd0 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:43.727378 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 25/50] 10.222.0.23 () {60 vars in 1140 bytes} [Thu Oct 13 12:56:43 2022] POST /identity/v3/projects => generated 298 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:56:45.852763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b8f43-00d1-45bb-a5b6-b2be9a44eed1 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:45.853293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b8f43-00d1-45bb-a5b6-b2be9a44eed1 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:45.853293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b8f43-00d1-45bb-a5b6-b2be9a44eed1 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:45.854623 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 26/51] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:56:45.859931 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a8f7e-05e2-4c5b-9c75-8c3bd5f45f98 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:45.860325 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a8f7e-05e2-4c5b-9c75-8c3bd5f45f98 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:45.860500 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a8f7e-05e2-4c5b-9c75-8c3bd5f45f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:45.878780 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-419a8f7e-05e2-4c5b-9c75-8c3bd5f45f98 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:56:45.880027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-419a8f7e-05e2-4c5b-9c75-8c3bd5f45f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:56:45.944597 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 26/52] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:56:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 13 12:56:46.215091 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b8f43-00d1-45bb-a5b6-b2be9a44eed1 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:46.311395 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b8f43-00d1-45bb-a5b6-b2be9a44eed1 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:46.327643 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:46.345538 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aebacd-6b42-44e6-ac84-d3f25563ce8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:46.346152 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aebacd-6b42-44e6-ac84-d3f25563ce8d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:46.346432 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aebacd-6b42-44e6-ac84-d3f25563ce8d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:46.346706 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aebacd-6b42-44e6-ac84-d3f25563ce8d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:46.347945 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aebacd-6b42-44e6-ac84-d3f25563ce8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:46.349391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aebacd-6b42-44e6-ac84-d3f25563ce8d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:46.350901 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 27/53] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:56:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:46.356243 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a8f7e-05e2-4c5b-9c75-8c3bd5f45f98 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:46.404204 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:46.452544 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a8f7e-05e2-4c5b-9c75-8c3bd5f45f98 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:46.485105 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270e2f0-3bb4-4c0b-9a3b-50edd06c9967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:46.485662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270e2f0-3bb4-4c0b-9a3b-50edd06c9967 None admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:46.485948 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270e2f0-3bb4-4c0b-9a3b-50edd06c9967 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:46.486223 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270e2f0-3bb4-4c0b-9a3b-50edd06c9967 None admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:46.486882 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2270e2f0-3bb4-4c0b-9a3b-50edd06c9967 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:46.488344 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2270e2f0-3bb4-4c0b-9a3b-50edd06c9967 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:46.513635 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 27/54] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 13 12:56:46 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:56:49.883672 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b31b2a-2369-45f5-bacb-2508b2260cfb None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:49.884712 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b31b2a-2369-45f5-bacb-2508b2260cfb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:49.885222 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b31b2a-2369-45f5-bacb-2508b2260cfb None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:49.886851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 28/55] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:56:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:56:49.892603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056adf65-2cc5-46c1-869b-13dec84b8331 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:49.893523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056adf65-2cc5-46c1-869b-13dec84b8331 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:49.894014 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056adf65-2cc5-46c1-869b-13dec84b8331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:49.909652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36465b-8abd-4950-9ce2-d65fa282318a None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:49.909652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36465b-8abd-4950-9ce2-d65fa282318a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:49.909652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36465b-8abd-4950-9ce2-d65fa282318a None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:49.909652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 29/56] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 13 12:56:49.915711 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5fd6a5-6432-48f5-98e7-2539e6207fb9 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:49.916270 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5fd6a5-6432-48f5-98e7-2539e6207fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:49.916736 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5fd6a5-6432-48f5-98e7-2539e6207fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:49.924072 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-056adf65-2cc5-46c1-869b-13dec84b8331 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:56:49.925558 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-056adf65-2cc5-46c1-869b-13dec84b8331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:56:50.025557 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 28/57] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:56:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:56:50.027351 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3a388-cc41-4cba-8138-f7b7316e07e1 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:50.027854 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3a388-cc41-4cba-8138-f7b7316e07e1 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:50.028323 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3a388-cc41-4cba-8138-f7b7316e07e1 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:50.029896 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 29/58] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:56:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:56:50.045038 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237182e-671e-498b-a2b1-e773b5dc5212 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:50.045668 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237182e-671e-498b-a2b1-e773b5dc5212 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:50.046132 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237182e-671e-498b-a2b1-e773b5dc5212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:50.080951 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-9237182e-671e-498b-a2b1-e773b5dc5212 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:56:50.082518 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-9237182e-671e-498b-a2b1-e773b5dc5212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:56:50.089891 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-ab5fd6a5-6432-48f5-98e7-2539e6207fb9 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:56:50.097370 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ab5fd6a5-6432-48f5-98e7-2539e6207fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:56:50.151611 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:50.202002 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 30/59] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:56:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 13 12:56:50.220330 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 30/60] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:56:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:56:50.263369 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e25b7-599b-48f1-bbbc-869b6a0827a8 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:50.264000 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e25b7-599b-48f1-bbbc-869b6a0827a8 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:50.264465 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e25b7-599b-48f1-bbbc-869b6a0827a8 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:50.266043 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 31/61] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:56:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:56:50.271475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13d36d-5cbb-42a0-aa73-c23a8dc48ea4 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:50.271975 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13d36d-5cbb-42a0-aa73-c23a8dc48ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:50.272425 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13d36d-5cbb-42a0-aa73-c23a8dc48ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:50.300791 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-5e13d36d-5cbb-42a0-aa73-c23a8dc48ea4 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:56:50.301964 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5e13d36d-5cbb-42a0-aa73-c23a8dc48ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:56:50.332056 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e25b7-599b-48f1-bbbc-869b6a0827a8 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:50.417386 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 31/62] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:56:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 13 12:56:50.472753 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e25b7-599b-48f1-bbbc-869b6a0827a8 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:50.512288 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13d36d-5cbb-42a0-aa73-c23a8dc48ea4 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:50.517822 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4652a4d-fc13-42fb-9902-c67cd87b7a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:50.518469 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4652a4d-fc13-42fb-9902-c67cd87b7a7f None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:50.518752 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4652a4d-fc13-42fb-9902-c67cd87b7a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:50.519026 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4652a4d-fc13-42fb-9902-c67cd87b7a7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:50.531229 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4652a4d-fc13-42fb-9902-c67cd87b7a7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:56:50.537292 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-f4652a4d-fc13-42fb-9902-c67cd87b7a7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:56:50.537292 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:56:50.537292 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:56:50.537292 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:56:50.537292 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:56:50.537292 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:56:50.537292 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:56:50.537292 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:56:50.537292 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:56:50.537292 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:56:50.537292 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:56:50.537292 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:56:50.537292 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:56:50.537292 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:56:50.537292 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:56:50.537292 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:56:50.538123 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:56:50.538123 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:56:50.538123 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:56:50.538123 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:56:50.538123 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:56:50.538123 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:56:50.538123 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:56:50.538123 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:56:50.538123 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:56:50.538123 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:56:50.538123 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:56:50.538123 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:56:50.538123 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:56:50.538123 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:56:50.538123 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:56:50.538123 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:56:50.539241 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:56:50.539241 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:56:50.539241 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:56:50.539241 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:56:50.539241 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:56:50.539241 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:56:50.539241 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:56:50.539241 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:56:50.539241 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:56:50.539241 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:56:50.539241 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:56:50.539241 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:56:50.539241 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 32/63] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 13 12:56:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:56:50.539241 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4652a4d-fc13-42fb-9902-c67cd87b7a7f None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:50.573794 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:50.576045 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:50.634276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13d36d-5cbb-42a0-aa73-c23a8dc48ea4 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:50.649899 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4652a4d-fc13-42fb-9902-c67cd87b7a7f None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:50.670712 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5211553-7c1f-43fe-ba34-702ab21b7d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:50.671947 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5211553-7c1f-43fe-ba34-702ab21b7d8c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:50.672293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5211553-7c1f-43fe-ba34-702ab21b7d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:50.672570 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5211553-7c1f-43fe-ba34-702ab21b7d8c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:50.683846 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5211553-7c1f-43fe-ba34-702ab21b7d8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:56:50.684210 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175c5b8-1c3f-4b66-94af-89e6ba2fd730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:50.684841 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175c5b8-1c3f-4b66-94af-89e6ba2fd730 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:50.685127 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175c5b8-1c3f-4b66-94af-89e6ba2fd730 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:50.685402 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175c5b8-1c3f-4b66-94af-89e6ba2fd730 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:50.689394 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-d5211553-7c1f-43fe-ba34-702ab21b7d8c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 13 12:56:50.689394 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:56:50.689394 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:56:50.689394 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:56:50.689394 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:56:50.689394 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:56:50.689394 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:56:50.689394 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:56:50.689394 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:56:50.689394 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:56:50.689394 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:56:50.689394 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:56:50.689394 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:56:50.689394 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:56:50.689394 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:56:50.689394 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:56:50.690196 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:56:50.690196 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:56:50.690196 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:56:50.690196 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:56:50.690196 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:56:50.690196 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:56:50.690196 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:56:50.690196 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:56:50.690196 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:56:50.690196 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:56:50.690196 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:56:50.690196 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:56:50.690196 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:56:50.690196 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:56:50.690196 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:56:50.690196 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:56:50.691700 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:56:50.691700 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:56:50.691700 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:56:50.691700 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:56:50.691700 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:56:50.691700 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:56:50.691700 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:56:50.691700 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:56:50.691700 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:56:50.691700 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:56:50.691700 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 13 12:56:50.691700 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:56:50.691700 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 32/64] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:56:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:56:50.692343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5211553-7c1f-43fe-ba34-702ab21b7d8c None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:50.703229 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175c5b8-1c3f-4b66-94af-89e6ba2fd730 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:56:50.708718 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-f175c5b8-1c3f-4b66-94af-89e6ba2fd730 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 13 12:56:50.708718 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:56:50.708718 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:56:50.708718 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:56:50.708718 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:56:50.708718 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:56:50.708718 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:56:50.708718 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:56:50.708718 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:56:50.708718 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:56:50.708718 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:56:50.708718 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:56:50.708718 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:56:50.708718 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:56:50.708718 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:56:50.708718 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:56:50.709535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:56:50.709535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:56:50.709535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:56:50.709535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:56:50.709535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:56:50.709535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:56:50.709535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:56:50.709535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:56:50.709535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:56:50.709535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:56:50.709535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:56:50.709535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:56:50.709535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:56:50.709535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:56:50.709535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:56:50.709535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:56:50.710283 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:56:50.710283 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:56:50.710283 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:56:50.710283 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:56:50.710283 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:56:50.710283 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:56:50.710283 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:56:50.710283 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:56:50.710283 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:56:50.710283 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:56:50.710283 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 13 12:56:50.710283 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:56:50.710283 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 33/65] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Oct 13 12:56:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:56:50.712125 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff654c0-c393-453d-8f0d-da2e87ec8252 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:50.712418 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff654c0-c393-453d-8f0d-da2e87ec8252 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:50.712694 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff654c0-c393-453d-8f0d-da2e87ec8252 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:50.714499 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 34/66] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:56:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:56:50.715229 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff654c0-c393-453d-8f0d-da2e87ec8252 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:50.795836 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5211553-7c1f-43fe-ba34-702ab21b7d8c None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:50.798590 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:50.800460 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:50.817474 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff654c0-c393-453d-8f0d-da2e87ec8252 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:50.830092 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6337ca-af73-4d85-be8e-2ab3ff64139f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:50.830700 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6337ca-af73-4d85-be8e-2ab3ff64139f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:50.830979 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6337ca-af73-4d85-be8e-2ab3ff64139f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:50.831282 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6337ca-af73-4d85-be8e-2ab3ff64139f None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:50.831971 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6337ca-af73-4d85-be8e-2ab3ff64139f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:56:50.832275 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6337ca-af73-4d85-be8e-2ab3ff64139f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:50.833649 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6337ca-af73-4d85-be8e-2ab3ff64139f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:50.841426 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 33/67] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:56:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:50.842207 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6337ca-af73-4d85-be8e-2ab3ff64139f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:50.851990 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f1d95b-f5d7-49ad-82f7-48ff1e004555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:50.852604 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1d95b-f5d7-49ad-82f7-48ff1e004555 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:50.852935 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1d95b-f5d7-49ad-82f7-48ff1e004555 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:50.853213 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1d95b-f5d7-49ad-82f7-48ff1e004555 None admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:50.853932 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f1d95b-f5d7-49ad-82f7-48ff1e004555 None admin] RBAC: Adding query filter params (name=member) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:56:50.854241 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f1d95b-f5d7-49ad-82f7-48ff1e004555 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:50.855668 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f1d95b-f5d7-49ad-82f7-48ff1e004555 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:50.864307 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 35/68] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 13 12:56:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:50.865072 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f1d95b-f5d7-49ad-82f7-48ff1e004555 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:50.948385 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6337ca-af73-4d85-be8e-2ab3ff64139f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:50.963426 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f1d95b-f5d7-49ad-82f7-48ff1e004555 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:50.982453 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65749da2-979a-4f0f-8065-b17ca53a4d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:50.983210 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65749da2-979a-4f0f-8065-b17ca53a4d32 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:50.983496 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65749da2-979a-4f0f-8065-b17ca53a4d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:50.984370 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65749da2-979a-4f0f-8065-b17ca53a4d32 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:50.994183 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65749da2-979a-4f0f-8065-b17ca53a4d32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:56:50.996092 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48264d5-91ce-44c4-804c-2d91b6698114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:50.996696 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48264d5-91ce-44c4-804c-2d91b6698114 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:50.996975 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48264d5-91ce-44c4-804c-2d91b6698114 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:50.997247 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48264d5-91ce-44c4-804c-2d91b6698114 None admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:50.997909 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48264d5-91ce-44c4-804c-2d91b6698114 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:56:50.998209 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48264d5-91ce-44c4-804c-2d91b6698114 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:50.999431 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-65749da2-979a-4f0f-8065-b17ca53a4d32 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 13 12:56:50.999431 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:56:50.999431 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:56:50.999431 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:56:50.999431 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:56:50.999431 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:56:50.999431 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:56:50.999431 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:56:50.999431 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:56:50.999431 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:56:50.999431 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:56:50.999431 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:56:50.999431 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:56:50.999431 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:56:50.999431 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:56:50.999431 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:56:51.000423 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:56:51.000423 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:56:51.000423 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:56:51.000423 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:56:51.000423 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:56:51.000423 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:56:51.000423 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:56:51.000423 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:56:51.000423 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:56:51.000423 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:56:51.000423 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:56:51.000423 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:56:51.000423 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:56:51.000423 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:56:51.000423 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:56:51.000423 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:56:51.003795 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:56:51.003795 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:56:51.003795 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:56:51.003795 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:56:51.003795 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:56:51.003795 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:56:51.003795 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:56:51.003795 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:56:51.003795 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:56:51.003795 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:56:51.003795 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 13 12:56:51.003795 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:56:51.004395 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48264d5-91ce-44c4-804c-2d91b6698114 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:51.004462 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 34/69] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:56:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:56:51.004462 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17d9f2-1df7-4917-a7e5-8dff062fae08 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:51.004462 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17d9f2-1df7-4917-a7e5-8dff062fae08 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:51.004462 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17d9f2-1df7-4917-a7e5-8dff062fae08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:51.012007 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 36/70] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:56:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:51.016043 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48264d5-91ce-44c4-804c-2d91b6698114 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:51.019409 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-9e17d9f2-1df7-4917-a7e5-8dff062fae08 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:56:51.020596 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9e17d9f2-1df7-4917-a7e5-8dff062fae08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:56:51.027855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:51.041250 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:51.088428 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 35/71] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:56:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:56:51.095941 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17d9f2-1df7-4917-a7e5-8dff062fae08 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:51.117406 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48264d5-91ce-44c4-804c-2d91b6698114 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:51.154318 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61307640-0466-40e7-a92d-0301571a92bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:51.154955 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61307640-0466-40e7-a92d-0301571a92bd None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:51.155290 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61307640-0466-40e7-a92d-0301571a92bd None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:51.156090 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61307640-0466-40e7-a92d-0301571a92bd None admin] PATH_INFO: `/v3/users/22269fc49ecd49a7b97d86f5937fc2dc` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:51.157233 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61307640-0466-40e7-a92d-0301571a92bd None admin] RBAC: Authorizing `identity:get_user(user_id=22269fc49ecd49a7b97d86f5937fc2dc)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:51.158692 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61307640-0466-40e7-a92d-0301571a92bd None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:51.160512 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 37/72] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:56:51 2022] GET /identity/v3/users/22269fc49ecd49a7b97d86f5937fc2dc => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:51.161074 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61307640-0466-40e7-a92d-0301571a92bd None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:51.207028 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17d9f2-1df7-4917-a7e5-8dff062fae08 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:51.242213 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:51.243944 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58045ff9-8713-40db-9916-efcc91d2868a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:51.244552 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58045ff9-8713-40db-9916-efcc91d2868a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:51.245004 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58045ff9-8713-40db-9916-efcc91d2868a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:51.245289 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58045ff9-8713-40db-9916-efcc91d2868a None admin] PATH_INFO: `/v3/users/a8e5d97876fd4bdfa107157ae3ae925b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:51.261986 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58045ff9-8713-40db-9916-efcc91d2868a None admin] RBAC: Authorizing `identity:get_user(user_id=a8e5d97876fd4bdfa107157ae3ae925b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:51.263476 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58045ff9-8713-40db-9916-efcc91d2868a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:51.263739 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61307640-0466-40e7-a92d-0301571a92bd None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:51.265913 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 36/73] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:56:51 2022] GET /identity/v3/users/a8e5d97876fd4bdfa107157ae3ae925b => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:51.266563 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58045ff9-8713-40db-9916-efcc91d2868a None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:51.283626 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:51.303718 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5ebca-bd00-4146-b440-6f46f7989511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:51.304361 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5ebca-bd00-4146-b440-6f46f7989511 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:51.304649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5ebca-bd00-4146-b440-6f46f7989511 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:51.304920 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5ebca-bd00-4146-b440-6f46f7989511 None admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:51.305588 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5ebca-bd00-4146-b440-6f46f7989511 None admin] RBAC: Adding query filter params (name=member) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:56:51.305891 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5ebca-bd00-4146-b440-6f46f7989511 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:51.307297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5ebca-bd00-4146-b440-6f46f7989511 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:51.315403 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 38/74] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 13 12:56:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:51.316196 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5ebca-bd00-4146-b440-6f46f7989511 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:51.375947 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58045ff9-8713-40db-9916-efcc91d2868a None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:51.409179 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2026ce-1972-4435-bba7-806c14575aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:51.409808 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2026ce-1972-4435-bba7-806c14575aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:51.410081 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2026ce-1972-4435-bba7-806c14575aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:51.410360 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2026ce-1972-4435-bba7-806c14575aa5 None admin] PATH_INFO: `/v3/users/a8e5d97876fd4bdfa107157ae3ae925b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:51.411523 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2026ce-1972-4435-bba7-806c14575aa5 None admin] RBAC: Authorizing `identity:get_user(user_id=a8e5d97876fd4bdfa107157ae3ae925b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:51.412971 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2026ce-1972-4435-bba7-806c14575aa5 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:51.414527 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 37/75] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:56:51 2022] GET /identity/v3/users/a8e5d97876fd4bdfa107157ae3ae925b => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:51.415315 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2026ce-1972-4435-bba7-806c14575aa5 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:51.416560 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5ebca-bd00-4146-b440-6f46f7989511 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:51.449392 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d683b4-5d6a-47fc-86fc-f3cbd5b52aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:51.450031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d683b4-5d6a-47fc-86fc-f3cbd5b52aaf None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:51.450316 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d683b4-5d6a-47fc-86fc-f3cbd5b52aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:51.450589 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d683b4-5d6a-47fc-86fc-f3cbd5b52aaf None admin] PATH_INFO: `/v3/projects/b14cb10007d440b4828ccc9869dc8aa2` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:51.451746 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d683b4-5d6a-47fc-86fc-f3cbd5b52aaf None admin] RBAC: Authorizing `identity:get_project(project_id=b14cb10007d440b4828ccc9869dc8aa2)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:51.453738 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d683b4-5d6a-47fc-86fc-f3cbd5b52aaf None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:51.455565 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 39/76] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 13 12:56:51 2022] GET /identity/v3/projects/b14cb10007d440b4828ccc9869dc8aa2 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:51.456183 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d683b4-5d6a-47fc-86fc-f3cbd5b52aaf None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:51.466785 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:51.498705 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:51.517836 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2026ce-1972-4435-bba7-806c14575aa5 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:51.550698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671cb7b3-9062-4188-b00b-656a60eaa9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:51.551350 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cb7b3-9062-4188-b00b-656a60eaa9a7 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:51.551633 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cb7b3-9062-4188-b00b-656a60eaa9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:51.551907 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cb7b3-9062-4188-b00b-656a60eaa9a7 None admin] PATH_INFO: `/v3/projects/b14cb10007d440b4828ccc9869dc8aa2` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:51.553034 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671cb7b3-9062-4188-b00b-656a60eaa9a7 None admin] RBAC: Authorizing `identity:get_project(project_id=b14cb10007d440b4828ccc9869dc8aa2)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:51.553711 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d683b4-5d6a-47fc-86fc-f3cbd5b52aaf None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:51.554536 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671cb7b3-9062-4188-b00b-656a60eaa9a7 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:51.556176 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 38/77] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 13 12:56:51 2022] GET /identity/v3/projects/b14cb10007d440b4828ccc9869dc8aa2 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:51.556938 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671cb7b3-9062-4188-b00b-656a60eaa9a7 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:51.589781 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baffad9-19cc-4c09-aa0d-a512c60dc59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:51.590407 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baffad9-19cc-4c09-aa0d-a512c60dc59f None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:51.590687 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baffad9-19cc-4c09-aa0d-a512c60dc59f None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:51.590961 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baffad9-19cc-4c09-aa0d-a512c60dc59f None admin] PATH_INFO: `/v3/users/a8e5d97876fd4bdfa107157ae3ae925b` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:51.592108 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baffad9-19cc-4c09-aa0d-a512c60dc59f None admin] RBAC: Authorizing `identity:get_user(user_id=a8e5d97876fd4bdfa107157ae3ae925b)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:51.594168 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baffad9-19cc-4c09-aa0d-a512c60dc59f None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:51.595919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 40/78] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:56:51 2022] GET /identity/v3/users/a8e5d97876fd4bdfa107157ae3ae925b => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:51.596600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baffad9-19cc-4c09-aa0d-a512c60dc59f None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:51.658793 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671cb7b3-9062-4188-b00b-656a60eaa9a7 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:51.670828 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:51.691811 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9e9de-d33c-4f57-8ef9-8cc02249adcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:51.692445 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9e9de-d33c-4f57-8ef9-8cc02249adcf None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:51.692754 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9e9de-d33c-4f57-8ef9-8cc02249adcf None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:51.693034 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9e9de-d33c-4f57-8ef9-8cc02249adcf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:51.694152 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d9e9de-d33c-4f57-8ef9-8cc02249adcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:51.694889 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baffad9-19cc-4c09-aa0d-a512c60dc59f None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:51.695614 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d9e9de-d33c-4f57-8ef9-8cc02249adcf None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:51.697165 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 39/79] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:56:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:51.701970 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9e9de-d33c-4f57-8ef9-8cc02249adcf None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:51.720944 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:51.728063 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda004c-5b24-4a22-ac62-3b3c882c0306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:51.728728 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda004c-5b24-4a22-ac62-3b3c882c0306 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:51.729012 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda004c-5b24-4a22-ac62-3b3c882c0306 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:51.729288 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda004c-5b24-4a22-ac62-3b3c882c0306 None admin] PATH_INFO: `/v3/projects/b14cb10007d440b4828ccc9869dc8aa2` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:51.730463 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda004c-5b24-4a22-ac62-3b3c882c0306 None admin] RBAC: Authorizing `identity:get_project(project_id=b14cb10007d440b4828ccc9869dc8aa2)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:51.731952 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda004c-5b24-4a22-ac62-3b3c882c0306 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:51.733605 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 41/80] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 13 12:56:51 2022] GET /identity/v3/projects/b14cb10007d440b4828ccc9869dc8aa2 => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:51.734358 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda004c-5b24-4a22-ac62-3b3c882c0306 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:51.800503 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9e9de-d33c-4f57-8ef9-8cc02249adcf None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:51.832715 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda004c-5b24-4a22-ac62-3b3c882c0306 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:51.834085 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96472e2-92b4-4896-904b-4164d22e953d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:51.834742 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96472e2-92b4-4896-904b-4164d22e953d None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:51.835027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96472e2-92b4-4896-904b-4164d22e953d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:51.835337 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96472e2-92b4-4896-904b-4164d22e953d None admin] PATH_INFO: `/v3/projects/b14cb10007d440b4828ccc9869dc8aa2/users/22269fc49ecd49a7b97d86f5937fc2dc/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:51.837263 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96472e2-92b4-4896-904b-4164d22e953d None admin] RBAC: Authorizing `identity:create_grant(project_id=b14cb10007d440b4828ccc9869dc8aa2, user_id=22269fc49ecd49a7b97d86f5937fc2dc, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:51.838718 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96472e2-92b4-4896-904b-4164d22e953d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:51.852797 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 40/81] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 13 12:56:51 2022] PUT /identity/v3/projects/b14cb10007d440b4828ccc9869dc8aa2/users/22269fc49ecd49a7b97d86f5937fc2dc/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 12:56:51.853411 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96472e2-92b4-4896-904b-4164d22e953d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:51.874056 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09cbff2-d7f0-4952-a6f0-04fc26d70de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:51.875397 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09cbff2-d7f0-4952-a6f0-04fc26d70de2 None admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:51.875680 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09cbff2-d7f0-4952-a6f0-04fc26d70de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:51.875952 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09cbff2-d7f0-4952-a6f0-04fc26d70de2 None admin] PATH_INFO: `/v3/projects/b14cb10007d440b4828ccc9869dc8aa2/users/a8e5d97876fd4bdfa107157ae3ae925b/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:51.877997 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09cbff2-d7f0-4952-a6f0-04fc26d70de2 None admin] RBAC: Authorizing `identity:create_grant(project_id=b14cb10007d440b4828ccc9869dc8aa2, user_id=a8e5d97876fd4bdfa107157ae3ae925b, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:51.879555 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09cbff2-d7f0-4952-a6f0-04fc26d70de2 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:51.894818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 42/82] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 13 12:56:51 2022] PUT /identity/v3/projects/b14cb10007d440b4828ccc9869dc8aa2/users/a8e5d97876fd4bdfa107157ae3ae925b/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 12:56:51.900094 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09cbff2-d7f0-4952-a6f0-04fc26d70de2 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:51.914962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:51.972875 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:52.008024 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96472e2-92b4-4896-904b-4164d22e953d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:52.039969 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09cbff2-d7f0-4952-a6f0-04fc26d70de2 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:52.059700 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39732567-4d96-48d4-b64f-dc0eaebb3ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:52.059700 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39732567-4d96-48d4-b64f-dc0eaebb3ee5 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:52.060886 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39732567-4d96-48d4-b64f-dc0eaebb3ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:52.060886 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39732567-4d96-48d4-b64f-dc0eaebb3ee5 None admin] PATH_INFO: `/v3/projects/7eeb9d9a51904ae08579decf06da2db6` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:52.063570 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39732567-4d96-48d4-b64f-dc0eaebb3ee5 None admin] RBAC: Authorizing `identity:get_project(project_id=7eeb9d9a51904ae08579decf06da2db6)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:52.064960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39732567-4d96-48d4-b64f-dc0eaebb3ee5 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:52.066851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 41/83] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 13 12:56:51 2022] GET /identity/v3/projects/7eeb9d9a51904ae08579decf06da2db6 => generated 312 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:52.067477 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39732567-4d96-48d4-b64f-dc0eaebb3ee5 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:52.085832 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9d316-90e5-466c-b7a8-6323ce7cb788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:52.086176 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9d316-90e5-466c-b7a8-6323ce7cb788 None admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:52.086495 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9d316-90e5-466c-b7a8-6323ce7cb788 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:52.086727 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9d316-90e5-466c-b7a8-6323ce7cb788 None admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:52.087418 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9d316-90e5-466c-b7a8-6323ce7cb788 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:52.088898 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9d316-90e5-466c-b7a8-6323ce7cb788 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:52.115032 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 43/84] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 13 12:56:51 2022] POST /identity/v3/users => generated 289 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:56:52.115715 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9d316-90e5-466c-b7a8-6323ce7cb788 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:52.221024 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:52.221352 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39732567-4d96-48d4-b64f-dc0eaebb3ee5 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:52.274959 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9d316-90e5-466c-b7a8-6323ce7cb788 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:52.302022 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e32a9-3107-41a6-a936-8c092aa3a3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:52.302707 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e32a9-3107-41a6-a936-8c092aa3a3d3 None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:52.302992 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e32a9-3107-41a6-a936-8c092aa3a3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:52.303292 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e32a9-3107-41a6-a936-8c092aa3a3d3 None admin] PATH_INFO: `/v3/projects/b14cb10007d440b4828ccc9869dc8aa2/users/a8e5d97876fd4bdfa107157ae3ae925b/roles/5a10c87b17414f04846918d777506811` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:52.309473 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4e32a9-3107-41a6-a936-8c092aa3a3d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=b14cb10007d440b4828ccc9869dc8aa2, user_id=a8e5d97876fd4bdfa107157ae3ae925b, role_id=5a10c87b17414f04846918d777506811)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:52.310944 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4e32a9-3107-41a6-a936-8c092aa3a3d3 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:52.334327 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 42/85] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 13 12:56:52 2022] PUT /identity/v3/projects/b14cb10007d440b4828ccc9869dc8aa2/users/a8e5d97876fd4bdfa107157ae3ae925b/roles/5a10c87b17414f04846918d777506811 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 12:56:52.339038 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f21a90-2010-440a-97e6-3a33b4563907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:52.339760 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f21a90-2010-440a-97e6-3a33b4563907 None admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:52.340042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f21a90-2010-440a-97e6-3a33b4563907 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:52.340315 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f21a90-2010-440a-97e6-3a33b4563907 None admin] PATH_INFO: `/v3/projects/7eeb9d9a51904ae08579decf06da2db6/users/a8e5d97876fd4bdfa107157ae3ae925b/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:52.342419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f21a90-2010-440a-97e6-3a33b4563907 None admin] RBAC: Authorizing `identity:create_grant(project_id=7eeb9d9a51904ae08579decf06da2db6, user_id=a8e5d97876fd4bdfa107157ae3ae925b, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:52.344701 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f21a90-2010-440a-97e6-3a33b4563907 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:52.362177 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 44/86] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 13 12:56:52 2022] PUT /identity/v3/projects/7eeb9d9a51904ae08579decf06da2db6/users/a8e5d97876fd4bdfa107157ae3ae925b/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 12:56:54.621028 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204866c2-8d6d-471d-b264-adea6a355385 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:54.621549 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204866c2-8d6d-471d-b264-adea6a355385 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:54.621855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204866c2-8d6d-471d-b264-adea6a355385 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:54.623328 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 43/87] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:56:54.629369 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415f0b7-8466-4b05-878f-a4498bbf3950 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:54.629761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415f0b7-8466-4b05-878f-a4498bbf3950 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:54.629952 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415f0b7-8466-4b05-878f-a4498bbf3950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:54.652352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-7415f0b7-8466-4b05-878f-a4498bbf3950 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:56:54.653558 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-7415f0b7-8466-4b05-878f-a4498bbf3950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:56:54.735003 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 45/88] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:56:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:56:54.948840 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5c415-905b-4cf1-b282-875ac8ceb982 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:54.949123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5c415-905b-4cf1-b282-875ac8ceb982 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:54.949814 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5c415-905b-4cf1-b282-875ac8ceb982 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:54.951675 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 44/89] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:56:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:56:54.958290 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e10be-2b77-4c12-8e46-366635ab78d5 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:54.958891 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e10be-2b77-4c12-8e46-366635ab78d5 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:54.959533 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e10be-2b77-4c12-8e46-366635ab78d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:54.992282 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-c60e10be-2b77-4c12-8e46-366635ab78d5 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:56:54.993870 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-c60e10be-2b77-4c12-8e46-366635ab78d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:56:55.059256 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:55.088927 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 46/90] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:56:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:56:55.089207 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a5c415-905b-4cf1-b282-875ac8ceb982 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:55.098662 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:55.109152 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff363163-c23b-4b29-9e0e-032416c7d7fe None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:55.109463 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff363163-c23b-4b29-9e0e-032416c7d7fe None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:55.109737 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff363163-c23b-4b29-9e0e-032416c7d7fe None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:55.111117 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 47/91] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:56:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 13 12:56:55.128812 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c444d84-98ce-4822-8f17-857c57a2b073 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:55.129106 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c444d84-98ce-4822-8f17-857c57a2b073 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:55.129385 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c444d84-98ce-4822-8f17-857c57a2b073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:55.160220 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-1c444d84-98ce-4822-8f17-857c57a2b073 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:56:55.161381 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-1c444d84-98ce-4822-8f17-857c57a2b073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:56:55.291705 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 48/92] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:56:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 13 12:56:55.292776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a5c415-905b-4cf1-b282-875ac8ceb982 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:55.345774 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8c3ad-3655-4a40-be5a-72051e05868a None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:55.346277 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8c3ad-3655-4a40-be5a-72051e05868a None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:55.346720 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8c3ad-3655-4a40-be5a-72051e05868a None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:55.348221 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 49/93] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:56:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 13 12:56:55.356531 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31630456-bfa2-415a-9d93-4abd70651b45 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:55.356657 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b9377d-d282-4cd8-a076-c6b4b5ad2e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:55.357095 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31630456-bfa2-415a-9d93-4abd70651b45 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:55.357340 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9377d-d282-4cd8-a076-c6b4b5ad2e7c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:55.357622 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9377d-d282-4cd8-a076-c6b4b5ad2e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:55.357793 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31630456-bfa2-415a-9d93-4abd70651b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:55.357943 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9377d-d282-4cd8-a076-c6b4b5ad2e7c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:55.370393 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b9377d-d282-4cd8-a076-c6b4b5ad2e7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:56:55.379480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-03b9377d-d282-4cd8-a076-c6b4b5ad2e7c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 13 12:56:55.379480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:56:55.379480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:56:55.379480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:56:55.379480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:56:55.379480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:56:55.379480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:56:55.379480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:56:55.379480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:56:55.379480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:56:55.379480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:56:55.379480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:56:55.379480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:56:55.379480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:56:55.379480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:56:55.379480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:56:55.380516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:56:55.380516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:56:55.380516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:56:55.380516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:56:55.380516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:56:55.380516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:56:55.380516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:56:55.380516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:56:55.380516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:56:55.380516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:56:55.380516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:56:55.380516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:56:55.380516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:56:55.380516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:56:55.380516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:56:55.380516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:56:55.381473 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:56:55.381473 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:56:55.381473 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:56:55.381473 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:56:55.381473 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:56:55.381473 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:56:55.381473 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:56:55.381473 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:56:55.381473 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:56:55.381473 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:56:55.381473 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 13 12:56:55.381473 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:56:55.383978 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 45/94] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:56:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:56:55.388021 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-31630456-bfa2-415a-9d93-4abd70651b45 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:56:55.389282 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-31630456-bfa2-415a-9d93-4abd70651b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:56:55.389419 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b9377d-d282-4cd8-a076-c6b4b5ad2e7c None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:55.486311 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:55.540889 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b9377d-d282-4cd8-a076-c6b4b5ad2e7c None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:55.558806 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 50/95] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:56:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:56:55.569742 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31630456-bfa2-415a-9d93-4abd70651b45 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:55.612652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec3bb0f-1201-44f7-8f0f-1246e08fb63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:55.613511 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec3bb0f-1201-44f7-8f0f-1246e08fb63a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:55.613927 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec3bb0f-1201-44f7-8f0f-1246e08fb63a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:55.614335 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec3bb0f-1201-44f7-8f0f-1246e08fb63a None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:55.615186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec3bb0f-1201-44f7-8f0f-1246e08fb63a None admin] RBAC: Adding query filter params (name=member) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:56:55.615642 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec3bb0f-1201-44f7-8f0f-1246e08fb63a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:55.617332 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec3bb0f-1201-44f7-8f0f-1246e08fb63a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:55.632537 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 46/96] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 13 12:56:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:56:55.638273 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec3bb0f-1201-44f7-8f0f-1246e08fb63a None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:55.652696 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:55.773982 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31630456-bfa2-415a-9d93-4abd70651b45 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:55.779047 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec3bb0f-1201-44f7-8f0f-1246e08fb63a None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:55.816198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:55.826665 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef843f00-3e66-43de-91b6-0d21cb96c9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:55.827596 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef843f00-3e66-43de-91b6-0d21cb96c9c6 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:55.828113 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef843f00-3e66-43de-91b6-0d21cb96c9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:55.828526 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef843f00-3e66-43de-91b6-0d21cb96c9c6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:55.837314 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a11e9-65ac-4a76-b01b-8ff27157d45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:55.838002 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a11e9-65ac-4a76-b01b-8ff27157d45a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:55.838294 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a11e9-65ac-4a76-b01b-8ff27157d45a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:55.838574 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a11e9-65ac-4a76-b01b-8ff27157d45a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:55.840306 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef843f00-3e66-43de-91b6-0d21cb96c9c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:56:55.845659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-ef843f00-3e66-43de-91b6-0d21cb96c9c6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 13 12:56:55.845659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:56:55.845659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:56:55.845659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:56:55.845659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:56:55.845659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:56:55.845659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:56:55.845659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:56:55.845659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:56:55.845659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:56:55.845659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:56:55.845659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:56:55.845659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:56:55.845659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:56:55.845659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:56:55.845659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:56:55.846681 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:56:55.846681 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:56:55.846681 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:56:55.846681 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:56:55.846681 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:56:55.846681 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:56:55.846681 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:56:55.846681 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:56:55.846681 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:56:55.846681 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:56:55.846681 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:56:55.846681 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:56:55.846681 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:56:55.846681 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:56:55.846681 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:56:55.846681 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:56:55.847706 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:56:55.847706 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:56:55.847706 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:56:55.847706 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:56:55.847706 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:56:55.847706 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:56:55.847706 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:56:55.847706 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:56:55.847706 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:56:55.847706 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:56:55.847706 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 13 12:56:55.847706 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:56:55.849666 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 51/97] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Oct 13 12:56:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:56:55.850596 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef843f00-3e66-43de-91b6-0d21cb96c9c6 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:55.860845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a11e9-65ac-4a76-b01b-8ff27157d45a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:56:55.866625 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-660a11e9-65ac-4a76-b01b-8ff27157d45a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 13 12:56:55.866625 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:56:55.866625 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:56:55.866625 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:56:55.866625 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:56:55.866625 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:56:55.866625 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:56:55.866625 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:56:55.866625 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:56:55.866625 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:56:55.866625 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:56:55.866625 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:56:55.866625 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:56:55.866625 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:56:55.866625 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:56:55.866625 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:56:55.867715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:56:55.867715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:56:55.867715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:56:55.867715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:56:55.867715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:56:55.867715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:56:55.867715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:56:55.867715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:56:55.867715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:56:55.867715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:56:55.867715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:56:55.867715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:56:55.867715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:56:55.867715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:56:55.867715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:56:55.867715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:56:55.868664 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:56:55.868664 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:56:55.868664 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:56:55.868664 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:56:55.868664 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:56:55.868664 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:56:55.868664 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:56:55.868664 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:56:55.868664 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:56:55.868664 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:56:55.868664 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 13 12:56:55.868664 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:56:55.875493 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 47/98] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:56:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:56:55.876326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a11e9-65ac-4a76-b01b-8ff27157d45a None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:55.971466 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:55.974573 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef843f00-3e66-43de-91b6-0d21cb96c9c6 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:56.011487 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15882a20-3204-4f76-ac9c-8debe0d73383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:56.012297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15882a20-3204-4f76-ac9c-8debe0d73383 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:56.012691 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15882a20-3204-4f76-ac9c-8debe0d73383 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:56.012971 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15882a20-3204-4f76-ac9c-8debe0d73383 None admin] PATH_INFO: `/v3/users/a8e5d97876fd4bdfa107157ae3ae925b` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:56.014163 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15882a20-3204-4f76-ac9c-8debe0d73383 None admin] RBAC: Authorizing `identity:get_user(user_id=a8e5d97876fd4bdfa107157ae3ae925b)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:56.016416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15882a20-3204-4f76-ac9c-8debe0d73383 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:56.018215 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 52/99] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:56:55 2022] GET /identity/v3/users/a8e5d97876fd4bdfa107157ae3ae925b => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:56.019066 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15882a20-3204-4f76-ac9c-8debe0d73383 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:56.022208 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a11e9-65ac-4a76-b01b-8ff27157d45a None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:56.060030 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13ce4e-fbd0-4c5c-a141-f94e4a840c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:56.060646 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13ce4e-fbd0-4c5c-a141-f94e4a840c78 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:56.060928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13ce4e-fbd0-4c5c-a141-f94e4a840c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:56.061204 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13ce4e-fbd0-4c5c-a141-f94e4a840c78 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:56.061909 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c13ce4e-fbd0-4c5c-a141-f94e4a840c78 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:56:56.062218 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c13ce4e-fbd0-4c5c-a141-f94e4a840c78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:56.063921 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c13ce4e-fbd0-4c5c-a141-f94e4a840c78 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:56.072808 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 48/100] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:56:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:56.073454 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13ce4e-fbd0-4c5c-a141-f94e4a840c78 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:56.127595 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:56.142687 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15882a20-3204-4f76-ac9c-8debe0d73383 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:56.180905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256a705-e00f-48ee-a6f4-f30af61889b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:56.181510 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256a705-e00f-48ee-a6f4-f30af61889b5 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:56.181793 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256a705-e00f-48ee-a6f4-f30af61889b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:56.182065 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256a705-e00f-48ee-a6f4-f30af61889b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:56.182734 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f256a705-e00f-48ee-a6f4-f30af61889b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:56:56.183036 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f256a705-e00f-48ee-a6f4-f30af61889b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:56.184212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13ce4e-fbd0-4c5c-a141-f94e4a840c78 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:56.184451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f256a705-e00f-48ee-a6f4-f30af61889b5 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:56.193370 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 53/101] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 13 12:56:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:56.194313 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256a705-e00f-48ee-a6f4-f30af61889b5 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:56.223998 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f18e48-cd41-40e5-bd8c-8ec612f5561f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:56.224574 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:56.224727 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f18e48-cd41-40e5-bd8c-8ec612f5561f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:56.224910 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f18e48-cd41-40e5-bd8c-8ec612f5561f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:56.225186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f18e48-cd41-40e5-bd8c-8ec612f5561f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:56.235211 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f18e48-cd41-40e5-bd8c-8ec612f5561f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:56:56.240425 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-e5f18e48-cd41-40e5-bd8c-8ec612f5561f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:56:56.240425 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:56:56.240425 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:56:56.240425 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:56:56.240425 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:56:56.240425 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:56:56.240425 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:56:56.240425 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:56:56.240425 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:56:56.240425 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:56:56.240425 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:56:56.240425 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:56:56.240425 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:56:56.240425 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:56:56.240425 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:56:56.240425 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:56:56.241416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:56:56.241416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:56:56.241416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:56:56.241416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:56:56.241416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:56:56.241416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:56:56.241416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:56:56.241416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:56:56.241416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:56:56.241416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:56:56.241416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:56:56.241416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:56:56.241416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:56:56.241416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:56:56.241416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:56:56.241416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:56:56.242168 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:56:56.242168 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:56:56.242168 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:56:56.242168 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:56:56.242168 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:56:56.242168 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:56:56.242168 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:56:56.242168 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:56:56.242168 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:56:56.242168 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:56:56.242168 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:56:56.242168 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:56:56.242168 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 49/102] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 13 12:56:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:56:56.242790 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095134a1-b150-4e2d-8c11-0d6dbdeb98b0 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:56.243620 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095134a1-b150-4e2d-8c11-0d6dbdeb98b0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:56.243620 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095134a1-b150-4e2d-8c11-0d6dbdeb98b0 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:56.244652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 50/103] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:56:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:56:56.245217 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095134a1-b150-4e2d-8c11-0d6dbdeb98b0 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:56.298532 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256a705-e00f-48ee-a6f4-f30af61889b5 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:56.332042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e434c-93e6-451b-8f18-44373e04d480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:56.332584 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e434c-93e6-451b-8f18-44373e04d480 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:56.332869 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e434c-93e6-451b-8f18-44373e04d480 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:56.333144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e434c-93e6-451b-8f18-44373e04d480 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:56.346129 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095134a1-b150-4e2d-8c11-0d6dbdeb98b0 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:56.356304 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:56.379627 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7544137-df6b-4111-b848-b4d6d0bd7e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:56.380257 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7544137-df6b-4111-b848-b4d6d0bd7e3f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:56.380537 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7544137-df6b-4111-b848-b4d6d0bd7e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:56.380815 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7544137-df6b-4111-b848-b4d6d0bd7e3f None admin] PATH_INFO: `/v3/users/a8e5d97876fd4bdfa107157ae3ae925b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:56.381956 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7544137-df6b-4111-b848-b4d6d0bd7e3f None admin] RBAC: Authorizing `identity:get_user(user_id=a8e5d97876fd4bdfa107157ae3ae925b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:56.383428 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7544137-df6b-4111-b848-b4d6d0bd7e3f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:56.384988 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 51/104] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:56:56 2022] GET /identity/v3/users/a8e5d97876fd4bdfa107157ae3ae925b => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:56.390231 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7544137-df6b-4111-b848-b4d6d0bd7e3f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:56.402888 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e434c-93e6-451b-8f18-44373e04d480 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:56.404382 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e434c-93e6-451b-8f18-44373e04d480 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:56.432164 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:56.484476 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7544137-df6b-4111-b848-b4d6d0bd7e3f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:56.507147 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 54/105] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 13 12:56:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 13 12:56:56.508253 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b99ff-0c67-46c6-9357-f391b5462cee None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:56.508544 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b99ff-0c67-46c6-9357-f391b5462cee None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:56.508824 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b99ff-0c67-46c6-9357-f391b5462cee None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:56.510091 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 55/106] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:56:56.515489 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed918d1-85bd-42f6-bc57-c299dfb05104 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:56.515787 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed918d1-85bd-42f6-bc57-c299dfb05104 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:56.516068 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed918d1-85bd-42f6-bc57-c299dfb05104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:56.526876 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aaefb3-ea2f-4225-a006-dc7b601df6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:56.527533 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aaefb3-ea2f-4225-a006-dc7b601df6f0 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:56.527815 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aaefb3-ea2f-4225-a006-dc7b601df6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:56.528090 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aaefb3-ea2f-4225-a006-dc7b601df6f0 None admin] PATH_INFO: `/v3/users/a8e5d97876fd4bdfa107157ae3ae925b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:56.529207 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aaefb3-ea2f-4225-a006-dc7b601df6f0 None admin] RBAC: Authorizing `identity:get_user(user_id=a8e5d97876fd4bdfa107157ae3ae925b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:56.530664 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aaefb3-ea2f-4225-a006-dc7b601df6f0 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:56.532397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 52/107] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:56:56 2022] GET /identity/v3/users/a8e5d97876fd4bdfa107157ae3ae925b => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:56.533076 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aaefb3-ea2f-4225-a006-dc7b601df6f0 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:56.537515 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-aed918d1-85bd-42f6-bc57-c299dfb05104 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:56:56.538944 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-aed918d1-85bd-42f6-bc57-c299dfb05104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:56:56.585807 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:56.606848 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 56/108] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:56:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:56:56.607755 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed918d1-85bd-42f6-bc57-c299dfb05104 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:56.660037 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aaefb3-ea2f-4225-a006-dc7b601df6f0 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:56.672921 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:56.698813 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2838f-da08-46d8-bc39-f4922f1ca409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:56.699455 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2838f-da08-46d8-bc39-f4922f1ca409 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:56.699741 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2838f-da08-46d8-bc39-f4922f1ca409 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:56.700015 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2838f-da08-46d8-bc39-f4922f1ca409 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:56.700688 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f2838f-da08-46d8-bc39-f4922f1ca409 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:56:56.700994 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f2838f-da08-46d8-bc39-f4922f1ca409 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:56.702404 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f2838f-da08-46d8-bc39-f4922f1ca409 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:56.711265 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 53/109] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:56:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:56.716787 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2838f-da08-46d8-bc39-f4922f1ca409 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:56.721735 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed918d1-85bd-42f6-bc57-c299dfb05104 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:56.763776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06411521-6f01-44f4-9590-dce7024830be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:56.764397 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06411521-6f01-44f4-9590-dce7024830be None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:56.764710 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06411521-6f01-44f4-9590-dce7024830be None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:56.765001 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06411521-6f01-44f4-9590-dce7024830be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:56.838770 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2838f-da08-46d8-bc39-f4922f1ca409 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:56.848606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06411521-6f01-44f4-9590-dce7024830be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:56.851899 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:56.852085 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06411521-6f01-44f4-9590-dce7024830be None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:56.879827 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854245c7-33c2-4999-a859-f07e664f6f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:56.880561 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854245c7-33c2-4999-a859-f07e664f6f23 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:56.880848 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854245c7-33c2-4999-a859-f07e664f6f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:56.881125 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854245c7-33c2-4999-a859-f07e664f6f23 None admin] PATH_INFO: `/v3/projects/b14cb10007d440b4828ccc9869dc8aa2` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:56.882249 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854245c7-33c2-4999-a859-f07e664f6f23 None admin] RBAC: Authorizing `identity:get_project(project_id=b14cb10007d440b4828ccc9869dc8aa2)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:56.883774 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854245c7-33c2-4999-a859-f07e664f6f23 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:56.885401 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 54/110] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 13 12:56:56 2022] GET /identity/v3/projects/b14cb10007d440b4828ccc9869dc8aa2 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:56.886488 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47c83c-9136-4ae0-bfb7-ed065a2173cd None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:56.886779 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47c83c-9136-4ae0-bfb7-ed065a2173cd None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:56.887092 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47c83c-9136-4ae0-bfb7-ed065a2173cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:56.904322 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8b47c83c-9136-4ae0-bfb7-ed065a2173cd None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:56:56.906073 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8b47c83c-9136-4ae0-bfb7-ed065a2173cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:56:56.911765 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:56.953309 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 57/111] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 13 12:56:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 13 12:56:56.954164 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06411521-6f01-44f4-9590-dce7024830be None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:56.973336 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 55/112] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:56:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:56:56.974324 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b47c83c-9136-4ae0-bfb7-ed065a2173cd None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:57.056464 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06411521-6f01-44f4-9590-dce7024830be None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:57.086097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b47c83c-9136-4ae0-bfb7-ed065a2173cd None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:57.092140 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56963fd-92e6-49be-ac44-3fe5ce3087eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:57.093216 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56963fd-92e6-49be-ac44-3fe5ce3087eb None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:57.093504 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56963fd-92e6-49be-ac44-3fe5ce3087eb None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:57.093783 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56963fd-92e6-49be-ac44-3fe5ce3087eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:57.094269 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:57.131099 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:57.132953 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c9b85-ca2b-4a97-b6c1-3f140424cd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:57.133709 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c9b85-ca2b-4a97-b6c1-3f140424cd02 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:57.133990 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c9b85-ca2b-4a97-b6c1-3f140424cd02 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:57.134424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c9b85-ca2b-4a97-b6c1-3f140424cd02 None admin] PATH_INFO: `/v3/users/22269fc49ecd49a7b97d86f5937fc2dc` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:57.135417 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c9b85-ca2b-4a97-b6c1-3f140424cd02 None admin] RBAC: Authorizing `identity:get_user(user_id=22269fc49ecd49a7b97d86f5937fc2dc)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:57.136847 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c9b85-ca2b-4a97-b6c1-3f140424cd02 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:57.139436 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 56/113] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:56:56 2022] GET /identity/v3/users/22269fc49ecd49a7b97d86f5937fc2dc => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:57.140080 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c9b85-ca2b-4a97-b6c1-3f140424cd02 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:57.175207 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56963fd-92e6-49be-ac44-3fe5ce3087eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:57.176998 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56963fd-92e6-49be-ac44-3fe5ce3087eb None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:57.240308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c9b85-ca2b-4a97-b6c1-3f140424cd02 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:57.274142 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86712298-7642-42f5-bec7-7bdbb7334fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:57.274793 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86712298-7642-42f5-bec7-7bdbb7334fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:57.275105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86712298-7642-42f5-bec7-7bdbb7334fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:57.275382 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86712298-7642-42f5-bec7-7bdbb7334fd5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:57.276889 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 58/114] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 13 12:56:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 13 12:56:57.282093 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56963fd-92e6-49be-ac44-3fe5ce3087eb None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:57.287868 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86712298-7642-42f5-bec7-7bdbb7334fd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:56:57.292693 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-86712298-7642-42f5-bec7-7bdbb7334fd5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 13 12:56:57.292693 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:56:57.292693 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:56:57.292693 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:56:57.292693 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:56:57.292693 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:56:57.292693 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:56:57.292693 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:56:57.292693 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:56:57.292693 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:56:57.292693 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:56:57.292693 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:56:57.292693 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:56:57.292693 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:56:57.292693 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:56:57.292693 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:56:57.293715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:56:57.293715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:56:57.293715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:56:57.293715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:56:57.293715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:56:57.293715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:56:57.293715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:56:57.293715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:56:57.293715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:56:57.293715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:56:57.293715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:56:57.293715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:56:57.293715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:56:57.293715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:56:57.293715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:56:57.293715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:56:57.295201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:56:57.295201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:56:57.295201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:56:57.295201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:56:57.295201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:56:57.295201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:56:57.295201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:56:57.295201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:56:57.295201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:56:57.295201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:56:57.295201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 13 12:56:57.295201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:56:57.295201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 57/115] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:56:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:56:57.295201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86712298-7642-42f5-bec7-7bdbb7334fd5 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:57.326577 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:57.338561 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:57.378102 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56963fd-92e6-49be-ac44-3fe5ce3087eb None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:57.391506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86712298-7642-42f5-bec7-7bdbb7334fd5 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:57.411965 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee5b5b-c94f-488d-93f8-a93de86aa402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:57.412573 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee5b5b-c94f-488d-93f8-a93de86aa402 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:57.412854 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee5b5b-c94f-488d-93f8-a93de86aa402 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:57.413128 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee5b5b-c94f-488d-93f8-a93de86aa402 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:57.413850 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee5b5b-c94f-488d-93f8-a93de86aa402 None admin] RBAC: Adding query filter params (role.id=10ba22bff4d24fd69e4c6def38370617, scope.project.id=b14cb10007d440b4828ccc9869dc8aa2, user.id=a8e5d97876fd4bdfa107157ae3ae925b) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:56:57.414161 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee5b5b-c94f-488d-93f8-a93de86aa402 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:57.415654 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee5b5b-c94f-488d-93f8-a93de86aa402 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:57.421336 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 59/116] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 13 12:56:57 2022] GET /identity/v3/role_assignments?user.id=a8e5d97876fd4bdfa107157ae3ae925b&scope.project.id=b14cb10007d440b4828ccc9869dc8aa2&role.id=10ba22bff4d24fd69e4c6def38370617 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:57.421925 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee5b5b-c94f-488d-93f8-a93de86aa402 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:57.429509 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d1d0c-9274-40c3-96c2-567232196e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:57.430171 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d1d0c-9274-40c3-96c2-567232196e66 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:57.430456 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d1d0c-9274-40c3-96c2-567232196e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:57.430734 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d1d0c-9274-40c3-96c2-567232196e66 None admin] PATH_INFO: `/v3/projects/b14cb10007d440b4828ccc9869dc8aa2` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:57.431989 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d1d0c-9274-40c3-96c2-567232196e66 None admin] RBAC: Authorizing `identity:get_project(project_id=b14cb10007d440b4828ccc9869dc8aa2)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:57.433475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d1d0c-9274-40c3-96c2-567232196e66 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:57.435239 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 58/117] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 13 12:56:57 2022] GET /identity/v3/projects/b14cb10007d440b4828ccc9869dc8aa2 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:57.440845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d1d0c-9274-40c3-96c2-567232196e66 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:57.535956 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee5b5b-c94f-488d-93f8-a93de86aa402 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:57.552493 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d1d0c-9274-40c3-96c2-567232196e66 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:57.554459 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:57.560714 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:57.572745 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d910c-28d1-4f1a-bb88-1083fba73076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:57.573375 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d910c-28d1-4f1a-bb88-1083fba73076 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:57.573655 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d910c-28d1-4f1a-bb88-1083fba73076 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:57.573928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d910c-28d1-4f1a-bb88-1083fba73076 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:57.594544 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d8167-1089-493d-8ad0-f6f7818aac5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:57.595316 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d8167-1089-493d-8ad0-f6f7818aac5e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:57.596252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d8167-1089-493d-8ad0-f6f7818aac5e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:57.596252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d8167-1089-493d-8ad0-f6f7818aac5e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:57.597086 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d8167-1089-493d-8ad0-f6f7818aac5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:57.598631 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d8167-1089-493d-8ad0-f6f7818aac5e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:57.600260 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 59/118] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:56:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:57.601104 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d8167-1089-493d-8ad0-f6f7818aac5e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:57.652169 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d910c-28d1-4f1a-bb88-1083fba73076 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:57.653573 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d910c-28d1-4f1a-bb88-1083fba73076 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:57.711248 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d8167-1089-493d-8ad0-f6f7818aac5e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:57.742885 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9bb71-5bba-4075-a4fe-d68f611e3b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:57.743535 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9bb71-5bba-4075-a4fe-d68f611e3b5c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:57.743817 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9bb71-5bba-4075-a4fe-d68f611e3b5c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:57.744090 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9bb71-5bba-4075-a4fe-d68f611e3b5c None admin] PATH_INFO: `/v3/projects/7eeb9d9a51904ae08579decf06da2db6` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:57.745207 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc9bb71-5bba-4075-a4fe-d68f611e3b5c None admin] RBAC: Authorizing `identity:get_project(project_id=7eeb9d9a51904ae08579decf06da2db6)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:57.746645 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc9bb71-5bba-4075-a4fe-d68f611e3b5c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:57.748289 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 60/119] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 13 12:56:57 2022] GET /identity/v3/projects/7eeb9d9a51904ae08579decf06da2db6 => generated 312 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:57.753811 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 60/120] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 13 12:56:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 13 12:56:57.754252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9bb71-5bba-4075-a4fe-d68f611e3b5c None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:57.754338 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d910c-28d1-4f1a-bb88-1083fba73076 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:57.776247 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:57.791119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:57.848557 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d910c-28d1-4f1a-bb88-1083fba73076 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:57.850007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9bb71-5bba-4075-a4fe-d68f611e3b5c None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:57.881432 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd85fe7-50f4-4e65-8a07-346724c6a7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:57.882069 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd85fe7-50f4-4e65-8a07-346724c6a7a4 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:57.882354 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd85fe7-50f4-4e65-8a07-346724c6a7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:57.882632 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd85fe7-50f4-4e65-8a07-346724c6a7a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:57.883394 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd85fe7-50f4-4e65-8a07-346724c6a7a4 None admin] RBAC: Adding query filter params (role.id=5a10c87b17414f04846918d777506811, scope.project.id=b14cb10007d440b4828ccc9869dc8aa2, user.id=a8e5d97876fd4bdfa107157ae3ae925b) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:56:57.883709 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd85fe7-50f4-4e65-8a07-346724c6a7a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:57.883824 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199785d7-7b3d-4c81-888f-fa67357fea62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:57.884419 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199785d7-7b3d-4c81-888f-fa67357fea62 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:57.884703 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199785d7-7b3d-4c81-888f-fa67357fea62 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:57.884980 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199785d7-7b3d-4c81-888f-fa67357fea62 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:57.885148 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd85fe7-50f4-4e65-8a07-346724c6a7a4 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:57.885639 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199785d7-7b3d-4c81-888f-fa67357fea62 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:56:57.885944 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199785d7-7b3d-4c81-888f-fa67357fea62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:57.887328 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199785d7-7b3d-4c81-888f-fa67357fea62 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:57.891408 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 61/121] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 13 12:56:57 2022] GET /identity/v3/role_assignments?user.id=a8e5d97876fd4bdfa107157ae3ae925b&scope.project.id=b14cb10007d440b4828ccc9869dc8aa2&role.id=5a10c87b17414f04846918d777506811 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:57.892175 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd85fe7-50f4-4e65-8a07-346724c6a7a4 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:57.895958 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 61/122] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 13 12:56:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:57.896838 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199785d7-7b3d-4c81-888f-fa67357fea62 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:58.006343 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:58.006900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:58.010238 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199785d7-7b3d-4c81-888f-fa67357fea62 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:58.019720 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd85fe7-50f4-4e65-8a07-346724c6a7a4 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:58.049481 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e5146-0912-4524-9f77-0c5eafedee42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:58.050207 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e5146-0912-4524-9f77-0c5eafedee42 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:58.050496 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e5146-0912-4524-9f77-0c5eafedee42 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:58.050772 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e5146-0912-4524-9f77-0c5eafedee42 None admin] PATH_INFO: `/v3/projects/b14cb10007d440b4828ccc9869dc8aa2` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:58.051971 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e5146-0912-4524-9f77-0c5eafedee42 None admin] RBAC: Authorizing `identity:get_project(project_id=b14cb10007d440b4828ccc9869dc8aa2)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:58.053430 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e5146-0912-4524-9f77-0c5eafedee42 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:58.055176 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 62/123] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 13 12:56:57 2022] GET /identity/v3/projects/b14cb10007d440b4828ccc9869dc8aa2 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:58.056962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3af63-58ae-43cd-ab94-0138bd846784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:58.058131 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3af63-58ae-43cd-ab94-0138bd846784 None admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:58.058131 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3af63-58ae-43cd-ab94-0138bd846784 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:58.058131 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c3af63-58ae-43cd-ab94-0138bd846784 None admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:58.058768 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c3af63-58ae-43cd-ab94-0138bd846784 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:58.060245 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c3af63-58ae-43cd-ab94-0138bd846784 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:58.060999 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e5146-0912-4524-9f77-0c5eafedee42 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:58.091776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 62/124] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Oct 13 12:56:57 2022] POST /identity/v3/projects => generated 302 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:56:58.091949 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3af63-58ae-43cd-ab94-0138bd846784 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:58.191564 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e5146-0912-4524-9f77-0c5eafedee42 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:58.220111 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c3af63-58ae-43cd-ab94-0138bd846784 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:58.226157 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9994b9d-d3bb-4641-92ea-821fcba70552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:58.226787 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9994b9d-d3bb-4641-92ea-821fcba70552 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:58.227097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9994b9d-d3bb-4641-92ea-821fcba70552 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:58.227379 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9994b9d-d3bb-4641-92ea-821fcba70552 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:58.228131 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9994b9d-d3bb-4641-92ea-821fcba70552 None admin] RBAC: Adding query filter params (role.id=10ba22bff4d24fd69e4c6def38370617, scope.project.id=7eeb9d9a51904ae08579decf06da2db6, user.id=a8e5d97876fd4bdfa107157ae3ae925b) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:56:58.228443 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9994b9d-d3bb-4641-92ea-821fcba70552 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:58.229935 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9994b9d-d3bb-4641-92ea-821fcba70552 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:58.239673 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 63/125] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 13 12:56:58 2022] GET /identity/v3/role_assignments?user.id=a8e5d97876fd4bdfa107157ae3ae925b&scope.project.id=7eeb9d9a51904ae08579decf06da2db6&role.id=10ba22bff4d24fd69e4c6def38370617 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:58.240584 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9994b9d-d3bb-4641-92ea-821fcba70552 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:58.268249 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db58cf12-fc37-4e73-b7b7-62d798c0f1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:58.268974 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db58cf12-fc37-4e73-b7b7-62d798c0f1d8 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:58.269301 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db58cf12-fc37-4e73-b7b7-62d798c0f1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:58.269715 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db58cf12-fc37-4e73-b7b7-62d798c0f1d8 None admin] PATH_INFO: `/v3/users/e484ffe15110483bb594a6add3e248c1` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:58.282697 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:58.283643 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:58.289112 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db58cf12-fc37-4e73-b7b7-62d798c0f1d8 None admin] RBAC: Authorizing `identity:get_user(user_id=e484ffe15110483bb594a6add3e248c1)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:58.290368 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db58cf12-fc37-4e73-b7b7-62d798c0f1d8 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:58.298540 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 63/126] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:56:58 2022] GET /identity/v3/users/e484ffe15110483bb594a6add3e248c1 => generated 289 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:58.298907 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db58cf12-fc37-4e73-b7b7-62d798c0f1d8 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:58.381777 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9994b9d-d3bb-4641-92ea-821fcba70552 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:58.424997 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f6f99-0f16-4c50-a861-16210c1b7294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:58.425616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f6f99-0f16-4c50-a861-16210c1b7294 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:58.425898 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f6f99-0f16-4c50-a861-16210c1b7294 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:58.426175 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f6f99-0f16-4c50-a861-16210c1b7294 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:58.426908 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2f6f99-0f16-4c50-a861-16210c1b7294 None admin] RBAC: Adding query filter params (role.id=70af94fc6a8149f2a38f74b843e1a9f0, scope.project.id=b14cb10007d440b4828ccc9869dc8aa2, user.id=22269fc49ecd49a7b97d86f5937fc2dc) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:56:58.427250 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2f6f99-0f16-4c50-a861-16210c1b7294 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:58.428761 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2f6f99-0f16-4c50-a861-16210c1b7294 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:58.433200 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db58cf12-fc37-4e73-b7b7-62d798c0f1d8 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:58.435993 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 64/127] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 13 12:56:58 2022] GET /identity/v3/role_assignments?user.id=22269fc49ecd49a7b97d86f5937fc2dc&scope.project.id=b14cb10007d440b4828ccc9869dc8aa2&role.id=70af94fc6a8149f2a38f74b843e1a9f0 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:58.477668 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fabebe6-4897-4210-a194-e7bd1137b087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:58.478601 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabebe6-4897-4210-a194-e7bd1137b087 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:58.478911 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabebe6-4897-4210-a194-e7bd1137b087 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:58.479293 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabebe6-4897-4210-a194-e7bd1137b087 None admin] PATH_INFO: `/v3/projects/b14cb10007d440b4828ccc9869dc8aa2` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:58.480574 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fabebe6-4897-4210-a194-e7bd1137b087 None admin] RBAC: Authorizing `identity:get_project(project_id=b14cb10007d440b4828ccc9869dc8aa2)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:58.482046 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fabebe6-4897-4210-a194-e7bd1137b087 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:58.484048 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 64/128] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 13 12:56:58 2022] GET /identity/v3/projects/b14cb10007d440b4828ccc9869dc8aa2 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:56:58.489973 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f6f99-0f16-4c50-a861-16210c1b7294 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:56:58.596866 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:56:58.604633 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f6f99-0f16-4c50-a861-16210c1b7294 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:56:58.640597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213ca09-5638-4c54-ad52-16fcf90e5d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:56:58.641259 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213ca09-5638-4c54-ad52-16fcf90e5d17 None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:56:58.641539 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213ca09-5638-4c54-ad52-16fcf90e5d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:56:58.641813 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213ca09-5638-4c54-ad52-16fcf90e5d17 None admin] PATH_INFO: `/v3/projects/b14cb10007d440b4828ccc9869dc8aa2/users/e484ffe15110483bb594a6add3e248c1/roles/4844dbf0f3e14acfaa7b2207a67377a7` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:56:58.643798 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d213ca09-5638-4c54-ad52-16fcf90e5d17 None admin] RBAC: Authorizing `identity:create_grant(project_id=b14cb10007d440b4828ccc9869dc8aa2, user_id=e484ffe15110483bb594a6add3e248c1, role_id=4844dbf0f3e14acfaa7b2207a67377a7)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:56:58.645257 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d213ca09-5638-4c54-ad52-16fcf90e5d17 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:56:58.659225 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 65/129] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 13 12:56:58 2022] PUT /identity/v3/projects/b14cb10007d440b4828ccc9869dc8aa2/users/e484ffe15110483bb594a6add3e248c1/roles/4844dbf0f3e14acfaa7b2207a67377a7 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 12:57:00.351602 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989eb7dc-7578-449c-8eb2-142a9499f4bf None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:00.352160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989eb7dc-7578-449c-8eb2-142a9499f4bf None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:00.352160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989eb7dc-7578-449c-8eb2-142a9499f4bf None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:00.353486 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 65/130] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:00.358653 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71a335-4bdd-4d55-af60-a3d54e444518 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:00.359082 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71a335-4bdd-4d55-af60-a3d54e444518 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:00.359273 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71a335-4bdd-4d55-af60-a3d54e444518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:00.376983 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-fd71a335-4bdd-4d55-af60-a3d54e444518 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:00.378152 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fd71a335-4bdd-4d55-af60-a3d54e444518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:00.441582 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 66/131] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:00.709789 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989eb7dc-7578-449c-8eb2-142a9499f4bf None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:00.757597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80f805-b331-4932-aa03-855e1b5c1583 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:00.757891 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80f805-b331-4932-aa03-855e1b5c1583 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:00.758167 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80f805-b331-4932-aa03-855e1b5c1583 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:00.759527 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 67/132] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 13 12:57:00.764357 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac7d31-b108-4f7f-8ffb-06915e07304e None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:00.765153 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac7d31-b108-4f7f-8ffb-06915e07304e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:00.765428 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac7d31-b108-4f7f-8ffb-06915e07304e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:00.774701 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:00.783854 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-11ac7d31-b108-4f7f-8ffb-06915e07304e None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:00.785053 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-11ac7d31-b108-4f7f-8ffb-06915e07304e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:00.811949 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989eb7dc-7578-449c-8eb2-142a9499f4bf None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:00.845748 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ddd075-3dfb-4658-956b-488dd5c16221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:00.846377 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ddd075-3dfb-4658-956b-488dd5c16221 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:00.846656 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ddd075-3dfb-4658-956b-488dd5c16221 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:00.846925 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ddd075-3dfb-4658-956b-488dd5c16221 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:00.848030 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ddd075-3dfb-4658-956b-488dd5c16221 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:00.849290 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 68/133] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 13 12:57:00.849465 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ddd075-3dfb-4658-956b-488dd5c16221 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:00.851280 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 66/134] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:00.856398 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ac7d31-b108-4f7f-8ffb-06915e07304e None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:00.873063 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:00.956721 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ac7d31-b108-4f7f-8ffb-06915e07304e None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:00.993271 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953bbb27-e195-4300-be4b-d8172924dc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:00.993816 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bbb27-e195-4300-be4b-d8172924dc31 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:00.994100 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bbb27-e195-4300-be4b-d8172924dc31 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:00.994375 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bbb27-e195-4300-be4b-d8172924dc31 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:00.995038 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953bbb27-e195-4300-be4b-d8172924dc31 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:00.996505 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953bbb27-e195-4300-be4b-d8172924dc31 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:01.022076 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 69/135] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 13 12:57:00 2022] POST /identity/v3/users => generated 283 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 12:57:01.117512 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ddd075-3dfb-4658-956b-488dd5c16221 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:01.220948 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ddd075-3dfb-4658-956b-488dd5c16221 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:01.254308 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed34226-15a8-47f5-af82-c372e4408a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:01.254876 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed34226-15a8-47f5-af82-c372e4408a85 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:01.255189 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed34226-15a8-47f5-af82-c372e4408a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:01.255464 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed34226-15a8-47f5-af82-c372e4408a85 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:01.257275 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:01.266641 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed34226-15a8-47f5-af82-c372e4408a85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:01.271919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-7ed34226-15a8-47f5-af82-c372e4408a85 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 13 12:57:01.271919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:01.271919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:01.271919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:01.271919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:01.271919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:01.271919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:01.271919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:01.271919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:01.271919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:01.271919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:01.271919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:01.271919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:01.271919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:01.271919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:01.271919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:01.272810 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:01.272810 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:01.272810 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:01.272810 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:01.272810 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:01.272810 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:01.272810 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:01.272810 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:01.272810 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:01.272810 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:01.272810 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:01.272810 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:01.272810 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:01.272810 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:01.272810 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:01.272810 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:01.273624 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:01.273624 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:01.273624 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:01.273624 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:01.273624 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:01.273624 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:01.273624 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:01.273624 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:01.273624 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:01.273624 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:01.273624 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 13 12:57:01.273624 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:01.273624 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 67/136] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:57:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:01.279125 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953bbb27-e195-4300-be4b-d8172924dc31 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:01.352291 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:01.388359 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953bbb27-e195-4300-be4b-d8172924dc31 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:01.424024 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b2e82-0f2a-46b8-942e-a425c0881eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:01.424643 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b2e82-0f2a-46b8-942e-a425c0881eab None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:01.425045 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b2e82-0f2a-46b8-942e-a425c0881eab None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:01.425189 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b2e82-0f2a-46b8-942e-a425c0881eab None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:01.425867 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b2e82-0f2a-46b8-942e-a425c0881eab None admin] RBAC: Adding query filter params (name=reader) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:01.426175 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b2e82-0f2a-46b8-942e-a425c0881eab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:01.427603 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b2e82-0f2a-46b8-942e-a425c0881eab None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:01.436830 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 70/137] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 13 12:57:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:01.442616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed34226-15a8-47f5-af82-c372e4408a85 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:01.552685 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed34226-15a8-47f5-af82-c372e4408a85 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:01.587500 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8af085-ea25-40a1-bff0-e85bebc1388f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:01.588143 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8af085-ea25-40a1-bff0-e85bebc1388f None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:01.588426 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8af085-ea25-40a1-bff0-e85bebc1388f None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:01.588705 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8af085-ea25-40a1-bff0-e85bebc1388f None admin] PATH_INFO: `/v3/users/e484ffe15110483bb594a6add3e248c1` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:01.589858 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8af085-ea25-40a1-bff0-e85bebc1388f None admin] RBAC: Authorizing `identity:get_user(user_id=e484ffe15110483bb594a6add3e248c1)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:01.591368 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8af085-ea25-40a1-bff0-e85bebc1388f None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:01.593803 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 68/138] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:57:01 2022] GET /identity/v3/users/e484ffe15110483bb594a6add3e248c1 => generated 289 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:01.599411 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b2e82-0f2a-46b8-942e-a425c0881eab None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:01.705833 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b2e82-0f2a-46b8-942e-a425c0881eab None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:01.740531 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9526b0c-2069-46e4-9529-5a0548a6f4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:01.741600 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9526b0c-2069-46e4-9529-5a0548a6f4da None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:01.742090 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9526b0c-2069-46e4-9529-5a0548a6f4da None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:01.742597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9526b0c-2069-46e4-9529-5a0548a6f4da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:01.745550 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:01.815967 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9526b0c-2069-46e4-9529-5a0548a6f4da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:01.817803 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9526b0c-2069-46e4-9529-5a0548a6f4da None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:01.919322 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 71/139] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 13 12:57:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 13 12:57:01.932170 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8af085-ea25-40a1-bff0-e85bebc1388f None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:01.997940 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:02.040570 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8af085-ea25-40a1-bff0-e85bebc1388f None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:02.075878 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208441e-9a3e-4a6c-a122-3115989579c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:02.076436 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208441e-9a3e-4a6c-a122-3115989579c9 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:02.077255 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208441e-9a3e-4a6c-a122-3115989579c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:02.077538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208441e-9a3e-4a6c-a122-3115989579c9 None admin] PATH_INFO: `/v3/projects/b14cb10007d440b4828ccc9869dc8aa2` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:02.078676 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5208441e-9a3e-4a6c-a122-3115989579c9 None admin] RBAC: Authorizing `identity:get_project(project_id=b14cb10007d440b4828ccc9869dc8aa2)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:02.080213 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5208441e-9a3e-4a6c-a122-3115989579c9 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:02.082145 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 69/140] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 13 12:57:01 2022] GET /identity/v3/projects/b14cb10007d440b4828ccc9869dc8aa2 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:02.087796 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9526b0c-2069-46e4-9529-5a0548a6f4da None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:02.122155 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:02.193267 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9526b0c-2069-46e4-9529-5a0548a6f4da None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:02.228396 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44c744-1812-4ecc-a062-b5ee040eec04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:02.229119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44c744-1812-4ecc-a062-b5ee040eec04 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:02.229404 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44c744-1812-4ecc-a062-b5ee040eec04 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:02.229682 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44c744-1812-4ecc-a062-b5ee040eec04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:02.230434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d44c744-1812-4ecc-a062-b5ee040eec04 None admin] RBAC: Adding query filter params (role.id=4844dbf0f3e14acfaa7b2207a67377a7, scope.project.id=b14cb10007d440b4828ccc9869dc8aa2, user.id=e484ffe15110483bb594a6add3e248c1) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:02.230750 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d44c744-1812-4ecc-a062-b5ee040eec04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:02.232302 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d44c744-1812-4ecc-a062-b5ee040eec04 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:02.239576 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 72/141] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 13 12:57:02 2022] GET /identity/v3/role_assignments?user.id=e484ffe15110483bb594a6add3e248c1&scope.project.id=b14cb10007d440b4828ccc9869dc8aa2&role.id=4844dbf0f3e14acfaa7b2207a67377a7 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:03.277369 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fec31-5f76-4cdb-a537-1bbafe29601e None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:03.277867 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fec31-5f76-4cdb-a537-1bbafe29601e None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:03.277867 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fec31-5f76-4cdb-a537-1bbafe29601e None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:03.279364 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 70/142] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:03.284913 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3556ea-3d71-4d1b-b6f0-ef4ccce2dc2b None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:03.285342 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3556ea-3d71-4d1b-b6f0-ef4ccce2dc2b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:03.285482 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3556ea-3d71-4d1b-b6f0-ef4ccce2dc2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:03.303552 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-ec3556ea-3d71-4d1b-b6f0-ef4ccce2dc2b None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:03.304734 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ec3556ea-3d71-4d1b-b6f0-ef4ccce2dc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:03.376898 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 73/143] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:03.449731 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735cb44-bb7a-412d-89c3-62b01129a2a0 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:03.450098 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735cb44-bb7a-412d-89c3-62b01129a2a0 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:03.450449 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735cb44-bb7a-412d-89c3-62b01129a2a0 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:03.452497 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 71/144] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:03.457602 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c0085-1b92-4692-b851-20d4fbf569e4 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:03.457885 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c0085-1b92-4692-b851-20d4fbf569e4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:03.458156 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c0085-1b92-4692-b851-20d4fbf569e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:03.476944 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0a0c0085-1b92-4692-b851-20d4fbf569e4 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:03.478118 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0a0c0085-1b92-4692-b851-20d4fbf569e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:03.501290 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:03.517516 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559fa193-efcb-4bf2-a4f8-454c4f5d107e None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:03.517807 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559fa193-efcb-4bf2-a4f8-454c4f5d107e None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:03.518068 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559fa193-efcb-4bf2-a4f8-454c4f5d107e None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:03.519568 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 72/145] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 13 12:57:03.524345 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379b94f-2d5a-452e-9c12-7e965f0ee5ed None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:03.524619 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379b94f-2d5a-452e-9c12-7e965f0ee5ed None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:03.524838 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379b94f-2d5a-452e-9c12-7e965f0ee5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:03.542989 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-8379b94f-2d5a-452e-9c12-7e965f0ee5ed None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:03.544124 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-8379b94f-2d5a-452e-9c12-7e965f0ee5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:03.548111 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 74/146] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:03.615160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 73/147] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 13 12:57:03.643898 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c0085-1b92-4692-b851-20d4fbf569e4 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:03.742341 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c0085-1b92-4692-b851-20d4fbf569e4 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:03.775945 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6397171-a391-4a2e-a97d-584d96d325a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:03.776564 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6397171-a391-4a2e-a97d-584d96d325a0 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:03.776833 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6397171-a391-4a2e-a97d-584d96d325a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:03.777102 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6397171-a391-4a2e-a97d-584d96d325a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:03.787036 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6397171-a391-4a2e-a97d-584d96d325a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:03.791787 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-b6397171-a391-4a2e-a97d-584d96d325a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:57:03.791787 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:03.791787 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:03.791787 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:03.791787 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:03.791787 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:03.791787 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:03.791787 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:03.791787 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:03.791787 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:03.791787 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:03.791787 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:03.791787 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:03.791787 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:03.791787 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:03.791787 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:03.792616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:03.792616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:03.792616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:03.792616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:03.792616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:03.792616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:03.792616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:03.792616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:03.792616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:03.792616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:03.792616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:03.792616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:03.792616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:03.792616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:03.792616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:03.792616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:03.793416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:03.793416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:03.793416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:03.793416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:03.793416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:03.793416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:03.793416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:03.793416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:03.793416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:03.793416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:03.793416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:57:03.793416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:03.795422 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 75/148] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 13 12:57:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:03.798716 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379b94f-2d5a-452e-9c12-7e965f0ee5ed None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:03.813404 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6397171-a391-4a2e-a97d-584d96d325a0 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:03.834650 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:03.863411 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:03.906544 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379b94f-2d5a-452e-9c12-7e965f0ee5ed None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:03.917187 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6397171-a391-4a2e-a97d-584d96d325a0 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:03.941338 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb08f0-684e-495c-a6f6-45e83d9ae741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:03.941954 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb08f0-684e-495c-a6f6-45e83d9ae741 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:03.942234 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb08f0-684e-495c-a6f6-45e83d9ae741 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:03.942510 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb08f0-684e-495c-a6f6-45e83d9ae741 None admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:03.943206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb08f0-684e-495c-a6f6-45e83d9ae741 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:03.943510 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb08f0-684e-495c-a6f6-45e83d9ae741 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:03.944885 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb08f0-684e-495c-a6f6-45e83d9ae741 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:03.950943 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370e444-115b-4ce6-aa88-19e213bdb323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:03.952189 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370e444-115b-4ce6-aa88-19e213bdb323 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:03.952482 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370e444-115b-4ce6-aa88-19e213bdb323 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:03.952753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370e444-115b-4ce6-aa88-19e213bdb323 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:03.953233 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 74/149] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:57:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:03.954017 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb08f0-684e-495c-a6f6-45e83d9ae741 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:03.967290 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2370e444-115b-4ce6-aa88-19e213bdb323 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:03.971999 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-2370e444-115b-4ce6-aa88-19e213bdb323 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 13 12:57:03.971999 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:03.971999 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:03.971999 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:03.971999 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:03.971999 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:03.971999 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:03.971999 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:03.971999 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:03.971999 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:03.971999 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:03.971999 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:03.971999 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:03.971999 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:03.971999 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:03.971999 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:03.972928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:03.972928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:03.972928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:03.972928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:03.972928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:03.972928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:03.972928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:03.972928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:03.972928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:03.972928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:03.972928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:03.972928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:03.972928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:03.972928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:03.972928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:03.972928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:03.973692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:03.973692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:03.973692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:03.973692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:03.973692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:03.973692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:03.973692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:03.973692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:03.973692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:03.973692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:03.973692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 13 12:57:03.973692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:03.973692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 76/150] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Oct 13 12:57:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:57:03.974431 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370e444-115b-4ce6-aa88-19e213bdb323 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:04.050113 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:04.051835 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb08f0-684e-495c-a6f6-45e83d9ae741 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:04.072420 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370e444-115b-4ce6-aa88-19e213bdb323 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:04.074224 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:04.085932 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c462d-fce4-4b82-a250-619f46fc1c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:04.086564 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c462d-fce4-4b82-a250-619f46fc1c57 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:04.086839 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c462d-fce4-4b82-a250-619f46fc1c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:04.087131 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c462d-fce4-4b82-a250-619f46fc1c57 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:04.088216 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5c462d-fce4-4b82-a250-619f46fc1c57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:04.089633 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5c462d-fce4-4b82-a250-619f46fc1c57 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:04.091203 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 75/151] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:04.096234 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c462d-fce4-4b82-a250-619f46fc1c57 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:04.105186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1415df27-805b-4389-b1d5-75821250c637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:04.105812 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415df27-805b-4389-b1d5-75821250c637 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:04.106088 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415df27-805b-4389-b1d5-75821250c637 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:04.106358 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415df27-805b-4389-b1d5-75821250c637 None admin] PATH_INFO: `/v3/users/8cc743a2781c4495aa0c4941fddd2739` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:04.120025 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1415df27-805b-4389-b1d5-75821250c637 None admin] RBAC: Authorizing `identity:get_user(user_id=8cc743a2781c4495aa0c4941fddd2739)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:04.121475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1415df27-805b-4389-b1d5-75821250c637 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:04.123073 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 77/152] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:57:03 2022] GET /identity/v3/users/8cc743a2781c4495aa0c4941fddd2739 => generated 283 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:04.123934 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1415df27-805b-4389-b1d5-75821250c637 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:04.192806 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c462d-fce4-4b82-a250-619f46fc1c57 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:04.222862 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1415df27-805b-4389-b1d5-75821250c637 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:04.225577 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705e358-c64e-40e7-8909-e732e45f9576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:04.226189 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705e358-c64e-40e7-8909-e732e45f9576 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:04.226472 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705e358-c64e-40e7-8909-e732e45f9576 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:04.226746 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705e358-c64e-40e7-8909-e732e45f9576 None admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:04.227447 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705e358-c64e-40e7-8909-e732e45f9576 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:04.227753 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705e358-c64e-40e7-8909-e732e45f9576 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:04.229136 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705e358-c64e-40e7-8909-e732e45f9576 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:04.237129 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 76/153] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:57:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:04.237129 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705e358-c64e-40e7-8909-e732e45f9576 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:04.256306 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff95b9b-03e4-4545-901b-4c1ccea1271d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:04.256914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff95b9b-03e4-4545-901b-4c1ccea1271d None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:04.257193 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff95b9b-03e4-4545-901b-4c1ccea1271d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:04.257464 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff95b9b-03e4-4545-901b-4c1ccea1271d None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:04.258139 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff95b9b-03e4-4545-901b-4c1ccea1271d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:04.259601 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff95b9b-03e4-4545-901b-4c1ccea1271d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:04.268863 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:04.282510 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 78/154] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 13 12:57:04 2022] POST /identity/v3/users => generated 297 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:57:04.295331 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff95b9b-03e4-4545-901b-4c1ccea1271d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:04.295872 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:04.346366 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705e358-c64e-40e7-8909-e732e45f9576 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:04.385384 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbce2d4-84ef-484c-becd-6aa522ed8f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:04.386227 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbce2d4-84ef-484c-becd-6aa522ed8f38 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:04.386516 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbce2d4-84ef-484c-becd-6aa522ed8f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:04.386792 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbce2d4-84ef-484c-becd-6aa522ed8f38 None admin] PATH_INFO: `/v3/projects/49ac31512d2943f2a28c22af2965292f` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:04.388211 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbce2d4-84ef-484c-becd-6aa522ed8f38 None admin] RBAC: Authorizing `identity:get_project(project_id=49ac31512d2943f2a28c22af2965292f)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:04.390330 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbce2d4-84ef-484c-becd-6aa522ed8f38 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:04.392360 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 77/155] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 13 12:57:04 2022] GET /identity/v3/projects/49ac31512d2943f2a28c22af2965292f => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:04.394559 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff95b9b-03e4-4545-901b-4c1ccea1271d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:04.398053 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbce2d4-84ef-484c-becd-6aa522ed8f38 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:04.432011 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c19ad-7682-4779-8f0f-e897fa6b2816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:04.432694 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c19ad-7682-4779-8f0f-e897fa6b2816 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:04.432938 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c19ad-7682-4779-8f0f-e897fa6b2816 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:04.433219 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c19ad-7682-4779-8f0f-e897fa6b2816 None admin] PATH_INFO: `/v3/users/8cc743a2781c4495aa0c4941fddd2739` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:04.434438 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430c19ad-7682-4779-8f0f-e897fa6b2816 None admin] RBAC: Authorizing `identity:get_user(user_id=8cc743a2781c4495aa0c4941fddd2739)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:04.436479 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430c19ad-7682-4779-8f0f-e897fa6b2816 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:04.438577 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 79/156] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:57:04 2022] GET /identity/v3/users/8cc743a2781c4495aa0c4941fddd2739 => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:04.444222 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c19ad-7682-4779-8f0f-e897fa6b2816 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:04.506737 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbce2d4-84ef-484c-becd-6aa522ed8f38 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:04.523419 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:04.533506 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:04.552216 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c374a3d-82e7-4f48-8fce-fedcb45e1c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:04.553257 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c374a3d-82e7-4f48-8fce-fedcb45e1c42 None admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:04.553559 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c374a3d-82e7-4f48-8fce-fedcb45e1c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:04.553825 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c374a3d-82e7-4f48-8fce-fedcb45e1c42 None admin] PATH_INFO: `/v3/projects/49ac31512d2943f2a28c22af2965292f/users/8cc743a2781c4495aa0c4941fddd2739/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:04.558896 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c374a3d-82e7-4f48-8fce-fedcb45e1c42 None admin] RBAC: Authorizing `identity:create_grant(project_id=49ac31512d2943f2a28c22af2965292f, user_id=8cc743a2781c4495aa0c4941fddd2739, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:04.560463 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c374a3d-82e7-4f48-8fce-fedcb45e1c42 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:04.579217 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c19ad-7682-4779-8f0f-e897fa6b2816 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:04.581193 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 78/157] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 13 12:57:04 2022] PUT /identity/v3/projects/49ac31512d2943f2a28c22af2965292f/users/8cc743a2781c4495aa0c4941fddd2739/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 12:57:04.623045 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84ee45-c444-4279-b906-65abd7d75040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:04.624254 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84ee45-c444-4279-b906-65abd7d75040 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:04.624535 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84ee45-c444-4279-b906-65abd7d75040 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:04.624881 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84ee45-c444-4279-b906-65abd7d75040 None admin] PATH_INFO: `/v3/projects/49ac31512d2943f2a28c22af2965292f` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:04.626117 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf84ee45-c444-4279-b906-65abd7d75040 None admin] RBAC: Authorizing `identity:get_project(project_id=49ac31512d2943f2a28c22af2965292f)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:04.627674 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf84ee45-c444-4279-b906-65abd7d75040 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:04.629431 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 80/158] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 13 12:57:04 2022] GET /identity/v3/projects/49ac31512d2943f2a28c22af2965292f => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:57:04.635797 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c374a3d-82e7-4f48-8fce-fedcb45e1c42 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:04.757046 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c374a3d-82e7-4f48-8fce-fedcb45e1c42 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:04.797105 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf021f-764d-458f-a7b0-3cd415c86c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:04.797827 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf021f-764d-458f-a7b0-3cd415c86c32 None admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:04.798104 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf021f-764d-458f-a7b0-3cd415c86c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:04.798422 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf021f-764d-458f-a7b0-3cd415c86c32 None admin] PATH_INFO: `/v3/projects/49ac31512d2943f2a28c22af2965292f/users/8cc743a2781c4495aa0c4941fddd2739/roles/5a10c87b17414f04846918d777506811` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:04.800658 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf021f-764d-458f-a7b0-3cd415c86c32 None admin] RBAC: Authorizing `identity:create_grant(project_id=49ac31512d2943f2a28c22af2965292f, user_id=8cc743a2781c4495aa0c4941fddd2739, role_id=5a10c87b17414f04846918d777506811)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:04.802094 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf021f-764d-458f-a7b0-3cd415c86c32 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:04.817433 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 79/159] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 13 12:57:04 2022] PUT /identity/v3/projects/49ac31512d2943f2a28c22af2965292f/users/8cc743a2781c4495aa0c4941fddd2739/roles/5a10c87b17414f04846918d777506811 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 12:57:06.588288 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00354e4-9ade-4b7a-bfde-2bd40ea24d30 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:06.588794 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00354e4-9ade-4b7a-bfde-2bd40ea24d30 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:06.588894 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00354e4-9ade-4b7a-bfde-2bd40ea24d30 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:06.590431 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 81/160] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:06.596178 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a39c6-0ee4-4044-aea0-463db32584aa None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:06.596578 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a39c6-0ee4-4044-aea0-463db32584aa None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:06.596744 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a39c6-0ee4-4044-aea0-463db32584aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:06.615499 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-821a39c6-0ee4-4044-aea0-463db32584aa None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:06.616690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-821a39c6-0ee4-4044-aea0-463db32584aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:06.651113 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:06.660246 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ef207-9ad7-49ed-802c-f3dbf8dfec33 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:06.660553 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ef207-9ad7-49ed-802c-f3dbf8dfec33 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:06.660824 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ef207-9ad7-49ed-802c-f3dbf8dfec33 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:06.662260 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 82/161] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 13 12:57:06.668056 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f7180-4878-492c-8c8f-345d9c12e511 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:06.668979 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f7180-4878-492c-8c8f-345d9c12e511 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:06.669455 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f7180-4878-492c-8c8f-345d9c12e511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:06.692214 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 80/162] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:06.693419 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-079f7180-4878-492c-8c8f-345d9c12e511 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:06.694773 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-079f7180-4878-492c-8c8f-345d9c12e511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:06.764765 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 83/163] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:06.802633 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d44d4-6872-4d91-a70c-86d85d09645b None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:06.802942 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d44d4-6872-4d91-a70c-86d85d09645b None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:06.803223 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d44d4-6872-4d91-a70c-86d85d09645b None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:06.804830 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 81/164] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:06.810114 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614a6fe-2af4-449c-9a03-40450e7a013a None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:06.810397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614a6fe-2af4-449c-9a03-40450e7a013a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:06.810671 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614a6fe-2af4-449c-9a03-40450e7a013a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:06.828871 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-9614a6fe-2af4-449c-9a03-40450e7a013a None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:06.830027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9614a6fe-2af4-449c-9a03-40450e7a013a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:06.885338 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:06.900123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 84/165] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:06.972058 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d44d4-6872-4d91-a70c-86d85d09645b None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:07.041066 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a0950-a106-4c58-82bb-a29047dc4946 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:07.041693 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a0950-a106-4c58-82bb-a29047dc4946 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:07.042167 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a0950-a106-4c58-82bb-a29047dc4946 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:07.045042 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 85/166] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 13 12:57:07.049723 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5f02d-b63c-440d-919a-84a6d4b393fb None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:07.050228 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5f02d-b63c-440d-919a-84a6d4b393fb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:07.050745 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5f02d-b63c-440d-919a-84a6d4b393fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:07.070031 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-e1a5f02d-b63c-440d-919a-84a6d4b393fb None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:07.071811 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e1a5f02d-b63c-440d-919a-84a6d4b393fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:07.084101 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d44d4-6872-4d91-a70c-86d85d09645b None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:07.121708 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1c709-7ef7-41e0-b01c-54d74a95b632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:07.122279 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1c709-7ef7-41e0-b01c-54d74a95b632 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:07.122560 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1c709-7ef7-41e0-b01c-54d74a95b632 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:07.122834 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1c709-7ef7-41e0-b01c-54d74a95b632 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:07.135615 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a1c709-7ef7-41e0-b01c-54d74a95b632 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:07.140187 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 86/167] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 13 12:57:07.140842 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5f02d-b63c-440d-919a-84a6d4b393fb None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:07.144144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-41a1c709-7ef7-41e0-b01c-54d74a95b632 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 13 12:57:07.144144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:07.144144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:07.144144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:07.144144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:07.144144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:07.144144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:07.144144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:07.144144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:07.144144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:07.144144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:07.144144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:07.144144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:07.144144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:07.144144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:07.144144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:07.145103 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:07.145103 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:07.145103 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:07.145103 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:07.145103 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:07.145103 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:07.145103 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:07.145103 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:07.145103 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:07.145103 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:07.145103 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:07.145103 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:07.145103 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:07.145103 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:07.145103 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:07.145103 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:07.145877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:07.145877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:07.145877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:07.145877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:07.145877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:07.145877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:07.145877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:07.145877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:07.145877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:07.145877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:07.145877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 13 12:57:07.145877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:07.145877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 82/168] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:57:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:07.155191 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1c709-7ef7-41e0-b01c-54d74a95b632 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:07.190716 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:07.250897 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5f02d-b63c-440d-919a-84a6d4b393fb None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:07.261986 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1c709-7ef7-41e0-b01c-54d74a95b632 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:07.268789 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:07.288379 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43699844-9559-4061-916e-7eae0800bd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:07.289327 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43699844-9559-4061-916e-7eae0800bd73 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:07.289615 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43699844-9559-4061-916e-7eae0800bd73 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:07.290146 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43699844-9559-4061-916e-7eae0800bd73 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:07.291274 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43699844-9559-4061-916e-7eae0800bd73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:07.292708 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43699844-9559-4061-916e-7eae0800bd73 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:07.294547 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 87/169] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:07.295692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43699844-9559-4061-916e-7eae0800bd73 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:07.298251 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140af6c-9488-4170-aab1-696f520d51d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:07.298860 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140af6c-9488-4170-aab1-696f520d51d2 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:07.299177 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140af6c-9488-4170-aab1-696f520d51d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:07.299674 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140af6c-9488-4170-aab1-696f520d51d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:07.300344 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140af6c-9488-4170-aab1-696f520d51d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:07.300643 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140af6c-9488-4170-aab1-696f520d51d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:07.302998 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140af6c-9488-4170-aab1-696f520d51d2 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:07.313353 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 83/170] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 13 12:57:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:57:07.313982 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140af6c-9488-4170-aab1-696f520d51d2 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:07.405906 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:07.408664 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43699844-9559-4061-916e-7eae0800bd73 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:07.417626 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140af6c-9488-4170-aab1-696f520d51d2 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:07.441949 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f48dc01-db42-492f-a88d-391f113fbc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:07.442567 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48dc01-db42-492f-a88d-391f113fbc3e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:07.442849 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48dc01-db42-492f-a88d-391f113fbc3e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:07.443149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48dc01-db42-492f-a88d-391f113fbc3e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:07.450834 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8fa88-2f1e-4ef8-ae1a-85380c107246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:07.451490 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8fa88-2f1e-4ef8-ae1a-85380c107246 None admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:07.451774 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8fa88-2f1e-4ef8-ae1a-85380c107246 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:07.452047 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8fa88-2f1e-4ef8-ae1a-85380c107246 None admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:07.452694 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b8fa88-2f1e-4ef8-ae1a-85380c107246 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:07.452992 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f48dc01-db42-492f-a88d-391f113fbc3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:07.454156 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b8fa88-2f1e-4ef8-ae1a-85380c107246 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:07.458147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-7f48dc01-db42-492f-a88d-391f113fbc3e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:57:07.458147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:07.458147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:07.458147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:07.458147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:07.458147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:07.458147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:07.458147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:07.458147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:07.458147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:07.458147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:07.458147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:07.458147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:07.458147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:07.458147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:07.458147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:07.459201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:07.459201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:07.459201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:07.459201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:07.459201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:07.459201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:07.459201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:07.459201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:07.459201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:07.459201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:07.459201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:07.459201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:07.459201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:07.459201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:07.459201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:07.459201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:07.464054 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:07.464054 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:07.464054 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:07.464054 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:07.464054 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:07.464054 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:07.464054 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:07.464054 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:07.464054 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:07.464054 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:07.464054 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:57:07.464054 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:07.464054 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 88/171] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 13 12:57:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:07.464054 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f48dc01-db42-492f-a88d-391f113fbc3e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:07.476791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 84/172] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 13 12:57:07 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:57:07.477606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8fa88-2f1e-4ef8-ae1a-85380c107246 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:07.513084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:07.587760 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f48dc01-db42-492f-a88d-391f113fbc3e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:07.595473 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8fa88-2f1e-4ef8-ae1a-85380c107246 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:07.628572 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e611a9-124c-46a7-88c2-c64c3e1e3a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:07.629224 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e611a9-124c-46a7-88c2-c64c3e1e3a02 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:07.629502 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e611a9-124c-46a7-88c2-c64c3e1e3a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:07.629783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e611a9-124c-46a7-88c2-c64c3e1e3a02 None admin] PATH_INFO: `/v3/users/a2622b0eb71446d88292fd57ae27fd0f` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:07.632386 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b25101-c108-4713-84ae-e81a984bbc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:07.633772 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b25101-c108-4713-84ae-e81a984bbc54 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:07.633772 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b25101-c108-4713-84ae-e81a984bbc54 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:07.633772 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b25101-c108-4713-84ae-e81a984bbc54 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:07.645026 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b25101-c108-4713-84ae-e81a984bbc54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:07.645482 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e611a9-124c-46a7-88c2-c64c3e1e3a02 None admin] RBAC: Authorizing `identity:get_user(user_id=a2622b0eb71446d88292fd57ae27fd0f)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:07.646967 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e611a9-124c-46a7-88c2-c64c3e1e3a02 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:07.648901 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 89/173] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:57:07 2022] GET /identity/v3/users/a2622b0eb71446d88292fd57ae27fd0f => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:07.649458 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e611a9-124c-46a7-88c2-c64c3e1e3a02 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:07.655466 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-f9b25101-c108-4713-84ae-e81a984bbc54 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 13 12:57:07.655466 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:07.655466 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:07.655466 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:07.655466 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:07.655466 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:07.655466 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:07.655466 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:07.655466 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:07.655466 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:07.655466 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:07.655466 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:07.655466 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:07.655466 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:07.655466 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:07.655466 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:07.656410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:07.656410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:07.656410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:07.656410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:07.656410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:07.656410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:07.656410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:07.656410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:07.656410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:07.656410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:07.656410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:07.656410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:07.656410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:07.656410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:07.656410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:07.656410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:07.657252 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:07.657252 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:07.657252 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:07.657252 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:07.657252 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:07.657252 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:07.657252 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:07.657252 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:07.657252 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:07.657252 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:07.657252 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 13 12:57:07.657252 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:07.662917 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 85/174] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Oct 13 12:57:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:07.662917 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b25101-c108-4713-84ae-e81a984bbc54 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:07.671050 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:07.777666 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e611a9-124c-46a7-88c2-c64c3e1e3a02 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:07.780635 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:07.784870 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b25101-c108-4713-84ae-e81a984bbc54 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:07.825955 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150a9b1-5614-4cdd-85d5-6be126800c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:07.827832 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150a9b1-5614-4cdd-85d5-6be126800c27 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:07.827832 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150a9b1-5614-4cdd-85d5-6be126800c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:07.827832 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150a9b1-5614-4cdd-85d5-6be126800c27 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:07.828073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0165879-a53c-4bc3-a89c-5b4e25ea6ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:07.828388 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150a9b1-5614-4cdd-85d5-6be126800c27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:07.828491 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0165879-a53c-4bc3-a89c-5b4e25ea6ab3 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:07.828679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0165879-a53c-4bc3-a89c-5b4e25ea6ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:07.828770 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150a9b1-5614-4cdd-85d5-6be126800c27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:07.828957 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0165879-a53c-4bc3-a89c-5b4e25ea6ab3 None admin] PATH_INFO: `/v3/projects/49ac31512d2943f2a28c22af2965292f` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:07.830064 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150a9b1-5614-4cdd-85d5-6be126800c27 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:07.830159 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0165879-a53c-4bc3-a89c-5b4e25ea6ab3 None admin] RBAC: Authorizing `identity:get_project(project_id=49ac31512d2943f2a28c22af2965292f)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:07.831866 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0165879-a53c-4bc3-a89c-5b4e25ea6ab3 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:07.833696 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 86/175] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 13 12:57:07 2022] GET /identity/v3/projects/49ac31512d2943f2a28c22af2965292f => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:07.835993 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0165879-a53c-4bc3-a89c-5b4e25ea6ab3 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:07.845405 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 90/176] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:57:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:07.846285 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150a9b1-5614-4cdd-85d5-6be126800c27 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:07.935969 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:07.962602 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0165879-a53c-4bc3-a89c-5b4e25ea6ab3 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:07.976551 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150a9b1-5614-4cdd-85d5-6be126800c27 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:08.007246 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a5b1a-85e9-4831-bf55-2bbbbfdce524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:08.007886 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a5b1a-85e9-4831-bf55-2bbbbfdce524 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:08.008169 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a5b1a-85e9-4831-bf55-2bbbbfdce524 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:08.008440 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a5b1a-85e9-4831-bf55-2bbbbfdce524 None admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:08.009111 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a5b1a-85e9-4831-bf55-2bbbbfdce524 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:08.009414 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a5b1a-85e9-4831-bf55-2bbbbfdce524 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:08.010786 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a5b1a-85e9-4831-bf55-2bbbbfdce524 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:08.016349 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4e9e6-b8cf-4844-b2a8-1ebef53a56d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:08.017844 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4e9e6-b8cf-4844-b2a8-1ebef53a56d7 None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:08.018334 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4e9e6-b8cf-4844-b2a8-1ebef53a56d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:08.019042 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4e9e6-b8cf-4844-b2a8-1ebef53a56d7 None admin] PATH_INFO: `/v3/projects/49ac31512d2943f2a28c22af2965292f/users/a2622b0eb71446d88292fd57ae27fd0f/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:08.025501 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 87/177] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:57:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:08.025501 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a5b1a-85e9-4831-bf55-2bbbbfdce524 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:08.026538 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d4e9e6-b8cf-4844-b2a8-1ebef53a56d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=49ac31512d2943f2a28c22af2965292f, user_id=a2622b0eb71446d88292fd57ae27fd0f, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:08.029155 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d4e9e6-b8cf-4844-b2a8-1ebef53a56d7 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:08.046416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 91/178] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 13 12:57:07 2022] PUT /identity/v3/projects/49ac31512d2943f2a28c22af2965292f/users/a2622b0eb71446d88292fd57ae27fd0f/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 12:57:08.047807 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4e9e6-b8cf-4844-b2a8-1ebef53a56d7 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:08.126341 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:08.151567 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a5b1a-85e9-4831-bf55-2bbbbfdce524 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:08.191615 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd293ab-d445-4d68-9059-cf1063362fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:08.192616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd293ab-d445-4d68-9059-cf1063362fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:08.193095 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd293ab-d445-4d68-9059-cf1063362fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:08.193547 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd293ab-d445-4d68-9059-cf1063362fe2 None admin] PATH_INFO: `/v3/users/8cc743a2781c4495aa0c4941fddd2739` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:08.194977 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd293ab-d445-4d68-9059-cf1063362fe2 None admin] RBAC: Authorizing `identity:get_user(user_id=8cc743a2781c4495aa0c4941fddd2739)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:08.196681 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd293ab-d445-4d68-9059-cf1063362fe2 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:08.198545 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 88/179] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:57:08 2022] GET /identity/v3/users/8cc743a2781c4495aa0c4941fddd2739 => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:08.208181 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd293ab-d445-4d68-9059-cf1063362fe2 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:08.215879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:08.277956 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4e9e6-b8cf-4844-b2a8-1ebef53a56d7 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:08.317400 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079fb5d9-4f4c-42a8-8cd2-56435e3e7300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:08.318715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079fb5d9-4f4c-42a8-8cd2-56435e3e7300 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:08.319210 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079fb5d9-4f4c-42a8-8cd2-56435e3e7300 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:08.319907 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079fb5d9-4f4c-42a8-8cd2-56435e3e7300 None admin] PATH_INFO: `/v3/users/8cc743a2781c4495aa0c4941fddd2739` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:08.321262 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079fb5d9-4f4c-42a8-8cd2-56435e3e7300 None admin] RBAC: Authorizing `identity:get_user(user_id=8cc743a2781c4495aa0c4941fddd2739)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:08.322907 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079fb5d9-4f4c-42a8-8cd2-56435e3e7300 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:08.324665 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 92/180] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:57:08 2022] GET /identity/v3/users/8cc743a2781c4495aa0c4941fddd2739 => generated 283 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:08.333738 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079fb5d9-4f4c-42a8-8cd2-56435e3e7300 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:08.342384 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd293ab-d445-4d68-9059-cf1063362fe2 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:08.383201 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df885e-f28a-4b52-85dc-ca3faa7a1606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:08.383866 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df885e-f28a-4b52-85dc-ca3faa7a1606 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:08.384150 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df885e-f28a-4b52-85dc-ca3faa7a1606 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:08.384423 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df885e-f28a-4b52-85dc-ca3faa7a1606 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:08.480258 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:08.490554 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079fb5d9-4f4c-42a8-8cd2-56435e3e7300 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:08.492509 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df885e-f28a-4b52-85dc-ca3faa7a1606 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:08.494352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df885e-f28a-4b52-85dc-ca3faa7a1606 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:08.503360 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:08.553792 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb3e449-57c6-4165-8c25-b58e4ca97607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:08.554130 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3e449-57c6-4165-8c25-b58e4ca97607 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:08.554404 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3e449-57c6-4165-8c25-b58e4ca97607 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:08.554686 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3e449-57c6-4165-8c25-b58e4ca97607 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:08.615608 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 89/181] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 13 12:57:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 13 12:57:08.621412 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df885e-f28a-4b52-85dc-ca3faa7a1606 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:08.658012 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb3e449-57c6-4165-8c25-b58e4ca97607 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:08.660802 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb3e449-57c6-4165-8c25-b58e4ca97607 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:08.784015 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df885e-f28a-4b52-85dc-ca3faa7a1606 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:08.798796 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:08.829587 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cf0953-3828-4b05-b1df-715440976252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:08.830579 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf0953-3828-4b05-b1df-715440976252 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:08.831076 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf0953-3828-4b05-b1df-715440976252 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:08.831540 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf0953-3828-4b05-b1df-715440976252 None admin] PATH_INFO: `/v3/projects/49ac31512d2943f2a28c22af2965292f` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:08.833284 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cf0953-3828-4b05-b1df-715440976252 None admin] RBAC: Authorizing `identity:get_project(project_id=49ac31512d2943f2a28c22af2965292f)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:08.835496 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cf0953-3828-4b05-b1df-715440976252 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:08.837357 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 90/182] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 13 12:57:08 2022] GET /identity/v3/projects/49ac31512d2943f2a28c22af2965292f => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:57:08.843870 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cf0953-3828-4b05-b1df-715440976252 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:08.851530 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:08.862761 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 93/183] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 13 12:57:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 13 12:57:08.869561 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb3e449-57c6-4165-8c25-b58e4ca97607 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:09.037717 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb3e449-57c6-4165-8c25-b58e4ca97607 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:09.054194 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cf0953-3828-4b05-b1df-715440976252 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:09.108443 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a990a-0d74-46a3-b324-5e50b958df08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:09.109076 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a990a-0d74-46a3-b324-5e50b958df08 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:09.109352 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a990a-0d74-46a3-b324-5e50b958df08 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:09.109622 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a990a-0d74-46a3-b324-5e50b958df08 None admin] PATH_INFO: `/v3/projects/49ac31512d2943f2a28c22af2965292f` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:09.112509 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126a990a-0d74-46a3-b324-5e50b958df08 None admin] RBAC: Authorizing `identity:get_project(project_id=49ac31512d2943f2a28c22af2965292f)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:09.113340 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126a990a-0d74-46a3-b324-5e50b958df08 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:09.114502 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 94/184] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 13 12:57:08 2022] GET /identity/v3/projects/49ac31512d2943f2a28c22af2965292f => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:09.118996 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1d2ca-91d2-4db6-9d0f-63a9c012c636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:09.119878 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1d2ca-91d2-4db6-9d0f-63a9c012c636 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:09.120353 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1d2ca-91d2-4db6-9d0f-63a9c012c636 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:09.120832 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1d2ca-91d2-4db6-9d0f-63a9c012c636 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:09.121945 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1d2ca-91d2-4db6-9d0f-63a9c012c636 None admin] RBAC: Adding query filter params (role.id=70af94fc6a8149f2a38f74b843e1a9f0, scope.project.id=49ac31512d2943f2a28c22af2965292f, user.id=8cc743a2781c4495aa0c4941fddd2739) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:09.122702 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1d2ca-91d2-4db6-9d0f-63a9c012c636 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:09.124606 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a990a-0d74-46a3-b324-5e50b958df08 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:09.125433 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1d2ca-91d2-4db6-9d0f-63a9c012c636 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:09.132081 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 91/185] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 13 12:57:08 2022] GET /identity/v3/role_assignments?user.id=8cc743a2781c4495aa0c4941fddd2739&scope.project.id=49ac31512d2943f2a28c22af2965292f&role.id=70af94fc6a8149f2a38f74b843e1a9f0 => generated 608 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:09.216456 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:09.260744 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a990a-0d74-46a3-b324-5e50b958df08 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:09.298516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b2f09-85f9-4f8a-a55d-8dc2b7e97ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:09.299173 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b2f09-85f9-4f8a-a55d-8dc2b7e97ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:09.299436 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b2f09-85f9-4f8a-a55d-8dc2b7e97ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:09.299704 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b2f09-85f9-4f8a-a55d-8dc2b7e97ce1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:09.300444 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b2f09-85f9-4f8a-a55d-8dc2b7e97ce1 None admin] RBAC: Adding query filter params (role.id=5a10c87b17414f04846918d777506811, scope.project.id=49ac31512d2943f2a28c22af2965292f, user.id=8cc743a2781c4495aa0c4941fddd2739) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:09.300748 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b2f09-85f9-4f8a-a55d-8dc2b7e97ce1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:09.302179 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b2f09-85f9-4f8a-a55d-8dc2b7e97ce1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:09.309118 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 95/186] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 13 12:57:09 2022] GET /identity/v3/role_assignments?user.id=8cc743a2781c4495aa0c4941fddd2739&scope.project.id=49ac31512d2943f2a28c22af2965292f&role.id=5a10c87b17414f04846918d777506811 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:09.919209 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8de52a-9689-42de-9d9a-8a724cb90bcd None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:09.919606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8de52a-9689-42de-9d9a-8a724cb90bcd None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:09.919825 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8de52a-9689-42de-9d9a-8a724cb90bcd None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:09.923384 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 92/187] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:09.928346 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6372e83-8912-44c2-9ba1-aa6dd4e98803 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:09.928757 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6372e83-8912-44c2-9ba1-aa6dd4e98803 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:09.928944 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6372e83-8912-44c2-9ba1-aa6dd4e98803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:09.947642 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-a6372e83-8912-44c2-9ba1-aa6dd4e98803 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:09.948814 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a6372e83-8912-44c2-9ba1-aa6dd4e98803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:10.015738 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 96/188] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:10.160994 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edb736-292c-401d-86ef-d89e75596c53 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:10.161295 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edb736-292c-401d-86ef-d89e75596c53 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:10.161475 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edb736-292c-401d-86ef-d89e75596c53 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:10.162992 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 93/189] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:10.167965 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2326232-4762-4a91-99a5-f8fa45ad07fc None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:10.168256 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2326232-4762-4a91-99a5-f8fa45ad07fc None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:10.168534 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2326232-4762-4a91-99a5-f8fa45ad07fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:10.187441 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f2326232-4762-4a91-99a5-f8fa45ad07fc None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:10.188872 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f2326232-4762-4a91-99a5-f8fa45ad07fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:10.237056 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:10.254391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 97/190] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:10.283128 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2edb736-292c-401d-86ef-d89e75596c53 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:10.324156 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:10.385406 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2edb736-292c-401d-86ef-d89e75596c53 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:10.418941 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f507c6bc-b71f-4f0d-9dda-c89088367e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:10.419545 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507c6bc-b71f-4f0d-9dda-c89088367e0b None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:10.419856 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507c6bc-b71f-4f0d-9dda-c89088367e0b None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:10.420728 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507c6bc-b71f-4f0d-9dda-c89088367e0b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:10.433098 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f507c6bc-b71f-4f0d-9dda-c89088367e0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:10.438644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-f507c6bc-b71f-4f0d-9dda-c89088367e0b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 13 12:57:10.438644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:10.438644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:10.438644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:10.438644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:10.438644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:10.438644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:10.438644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:10.438644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:10.438644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:10.438644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:10.438644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:10.438644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:10.438644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:10.438644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:10.438644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:10.439648 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:10.439648 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:10.439648 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:10.439648 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:10.439648 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:10.439648 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:10.439648 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:10.439648 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:10.439648 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:10.439648 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:10.439648 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:10.439648 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:10.439648 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:10.439648 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:10.439648 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:10.439648 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:10.440410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:10.440410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:10.440410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:10.440410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:10.440410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:10.440410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:10.440410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:10.440410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:10.440410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:10.440410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:10.440410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 13 12:57:10.440410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:10.443459 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 94/191] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:57:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:10.445748 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2326232-4762-4a91-99a5-f8fa45ad07fc None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:10.523606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f507c6bc-b71f-4f0d-9dda-c89088367e0b None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:10.544391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2326232-4762-4a91-99a5-f8fa45ad07fc None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:10.577085 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c485731-47fe-48e4-b0f3-c68a4c49ca65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:10.577686 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c485731-47fe-48e4-b0f3-c68a4c49ca65 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:10.577958 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c485731-47fe-48e4-b0f3-c68a4c49ca65 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:10.578225 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c485731-47fe-48e4-b0f3-c68a4c49ca65 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:10.578882 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c485731-47fe-48e4-b0f3-c68a4c49ca65 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:10.579213 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c485731-47fe-48e4-b0f3-c68a4c49ca65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:10.580574 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c485731-47fe-48e4-b0f3-c68a4c49ca65 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:10.588464 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 98/192] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 13 12:57:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:10.589673 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f050b44-e2c4-4dfe-bfb0-db7ce2867403 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:10.589954 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f050b44-e2c4-4dfe-bfb0-db7ce2867403 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:10.590222 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f050b44-e2c4-4dfe-bfb0-db7ce2867403 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:10.591440 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 99/193] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:10.594280 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f050b44-e2c4-4dfe-bfb0-db7ce2867403 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:10.624419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:10.628507 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f507c6bc-b71f-4f0d-9dda-c89088367e0b None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:10.645356 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:10.662158 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a6ad49-c2e8-4bc8-9235-7a03c136adb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:10.662783 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6ad49-c2e8-4bc8-9235-7a03c136adb0 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:10.663065 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6ad49-c2e8-4bc8-9235-7a03c136adb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:10.663355 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6ad49-c2e8-4bc8-9235-7a03c136adb0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:10.664450 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a6ad49-c2e8-4bc8-9235-7a03c136adb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:10.665900 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a6ad49-c2e8-4bc8-9235-7a03c136adb0 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:10.667498 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 95/194] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:57:10.668654 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe99eae-e264-4b66-bcf9-0d7620742993 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:10.668932 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe99eae-e264-4b66-bcf9-0d7620742993 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:10.669248 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe99eae-e264-4b66-bcf9-0d7620742993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:10.687076 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-2fe99eae-e264-4b66-bcf9-0d7620742993 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:10.688511 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-2fe99eae-e264-4b66-bcf9-0d7620742993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:10.700399 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f050b44-e2c4-4dfe-bfb0-db7ce2867403 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:10.733389 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70631d-42bf-400c-961b-2e3e4be56aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:10.734007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70631d-42bf-400c-961b-2e3e4be56aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:10.734282 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70631d-42bf-400c-961b-2e3e4be56aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:10.734551 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70631d-42bf-400c-961b-2e3e4be56aa1 None admin] PATH_INFO: `/v3/users/1efc885e761842dfad53f92df5a4c4ba` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:10.746881 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70631d-42bf-400c-961b-2e3e4be56aa1 None admin] RBAC: Authorizing `identity:get_user(user_id=1efc885e761842dfad53f92df5a4c4ba)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:10.748356 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70631d-42bf-400c-961b-2e3e4be56aa1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:10.749920 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 100/195] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:57:10 2022] GET /identity/v3/users/1efc885e761842dfad53f92df5a4c4ba => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:57:10.750725 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70631d-42bf-400c-961b-2e3e4be56aa1 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:10.757873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 96/196] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:10.758691 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe99eae-e264-4b66-bcf9-0d7620742993 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:10.855100 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:10.858419 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70631d-42bf-400c-961b-2e3e4be56aa1 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:10.861656 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe99eae-e264-4b66-bcf9-0d7620742993 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:10.867298 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:10.896548 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce34cc2-8fe5-4bcf-ad0e-f62ff99bad46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:10.896713 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d366b125-3fc5-4b77-bf23-5e07756670e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:10.897154 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce34cc2-8fe5-4bcf-ad0e-f62ff99bad46 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:10.897256 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366b125-3fc5-4b77-bf23-5e07756670e5 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:10.897420 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce34cc2-8fe5-4bcf-ad0e-f62ff99bad46 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:10.897517 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366b125-3fc5-4b77-bf23-5e07756670e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:10.897676 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce34cc2-8fe5-4bcf-ad0e-f62ff99bad46 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:10.897768 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366b125-3fc5-4b77-bf23-5e07756670e5 None admin] PATH_INFO: `/v3/projects/49ac31512d2943f2a28c22af2965292f` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:10.898321 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce34cc2-8fe5-4bcf-ad0e-f62ff99bad46 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:10.898849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d366b125-3fc5-4b77-bf23-5e07756670e5 None admin] RBAC: Authorizing `identity:get_project(project_id=49ac31512d2943f2a28c22af2965292f)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:10.899770 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce34cc2-8fe5-4bcf-ad0e-f62ff99bad46 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:10.900322 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d366b125-3fc5-4b77-bf23-5e07756670e5 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:10.902164 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 97/197] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 13 12:57:10 2022] GET /identity/v3/projects/49ac31512d2943f2a28c22af2965292f => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:10.908194 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d366b125-3fc5-4b77-bf23-5e07756670e5 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:10.926598 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 101/198] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 13 12:57:10 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:57:11.021628 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d366b125-3fc5-4b77-bf23-5e07756670e5 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:11.031336 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce34cc2-8fe5-4bcf-ad0e-f62ff99bad46 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:11.063482 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0441c92-9f12-470c-8aca-21839ed1603f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:11.064160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0441c92-9f12-470c-8aca-21839ed1603f None admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:11.064437 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0441c92-9f12-470c-8aca-21839ed1603f None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:11.065023 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0441c92-9f12-470c-8aca-21839ed1603f None admin] PATH_INFO: `/v3/projects/49ac31512d2943f2a28c22af2965292f/users/1efc885e761842dfad53f92df5a4c4ba/roles/4844dbf0f3e14acfaa7b2207a67377a7` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:11.066673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0441c92-9f12-470c-8aca-21839ed1603f None admin] RBAC: Authorizing `identity:create_grant(project_id=49ac31512d2943f2a28c22af2965292f, user_id=1efc885e761842dfad53f92df5a4c4ba, role_id=4844dbf0f3e14acfaa7b2207a67377a7)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:11.068168 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0441c92-9f12-470c-8aca-21839ed1603f None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:11.082879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 98/199] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 13 12:57:10 2022] PUT /identity/v3/projects/49ac31512d2943f2a28c22af2965292f/users/1efc885e761842dfad53f92df5a4c4ba/roles/4844dbf0f3e14acfaa7b2207a67377a7 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 12:57:11.144407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce34cc2-8fe5-4bcf-ad0e-f62ff99bad46 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:11.182940 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc51a4-2044-44f6-9b77-f61fd1d8bf06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:11.184008 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc51a4-2044-44f6-9b77-f61fd1d8bf06 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:11.184202 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc51a4-2044-44f6-9b77-f61fd1d8bf06 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:11.184540 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc51a4-2044-44f6-9b77-f61fd1d8bf06 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:11.198989 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc51a4-2044-44f6-9b77-f61fd1d8bf06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:11.205297 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-5bdc51a4-2044-44f6-9b77-f61fd1d8bf06 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 13 12:57:11.205297 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:11.205297 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:11.205297 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:11.205297 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:11.205297 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:11.205297 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:11.205297 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:11.205297 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:11.205297 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:11.205297 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:11.205297 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:11.205297 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:11.205297 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:11.205297 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:11.205297 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:11.206434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:11.206434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:11.206434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:11.206434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:11.206434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:11.206434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:11.206434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:11.206434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:11.206434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:11.206434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:11.206434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:11.206434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:11.206434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:11.206434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:11.206434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:11.206434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:11.207493 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:11.207493 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:11.207493 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:11.207493 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:11.207493 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:11.207493 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:11.207493 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:11.207493 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:11.207493 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:11.207493 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:11.207493 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 13 12:57:11.207493 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:11.207493 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 102/200] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:57:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:57:11.212957 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0441c92-9f12-470c-8aca-21839ed1603f None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:11.256999 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:11.341859 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0441c92-9f12-470c-8aca-21839ed1603f None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:11.377370 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec9c0c-37ab-4b6b-a824-ab430f34a87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:11.378042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec9c0c-37ab-4b6b-a824-ab430f34a87e None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:11.378321 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec9c0c-37ab-4b6b-a824-ab430f34a87e None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:11.378588 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec9c0c-37ab-4b6b-a824-ab430f34a87e None admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:11.379330 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baec9c0c-37ab-4b6b-a824-ab430f34a87e None admin] RBAC: Adding query filter params (name=member) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:11.379632 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baec9c0c-37ab-4b6b-a824-ab430f34a87e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:11.380993 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baec9c0c-37ab-4b6b-a824-ab430f34a87e None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:11.390165 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 99/201] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 13 12:57:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:11.395890 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc51a4-2044-44f6-9b77-f61fd1d8bf06 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:11.421007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:11.497353 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc51a4-2044-44f6-9b77-f61fd1d8bf06 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:11.530812 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9b523-7797-41ef-bc37-f8870b8afbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:11.531543 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9b523-7797-41ef-bc37-f8870b8afbb4 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:11.531819 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9b523-7797-41ef-bc37-f8870b8afbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:11.532089 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9b523-7797-41ef-bc37-f8870b8afbb4 None admin] PATH_INFO: `/v3/users/a2622b0eb71446d88292fd57ae27fd0f` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:11.533225 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd9b523-7797-41ef-bc37-f8870b8afbb4 None admin] RBAC: Authorizing `identity:get_user(user_id=a2622b0eb71446d88292fd57ae27fd0f)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:11.534658 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd9b523-7797-41ef-bc37-f8870b8afbb4 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:11.540823 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 103/202] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:57:11 2022] GET /identity/v3/users/a2622b0eb71446d88292fd57ae27fd0f => generated 297 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:11.542184 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec9c0c-37ab-4b6b-a824-ab430f34a87e None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:11.635785 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:11.652365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec9c0c-37ab-4b6b-a824-ab430f34a87e None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:11.691367 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9114e4-6fc5-410b-a347-362d577843ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:11.691981 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9114e4-6fc5-410b-a347-362d577843ba None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:11.692261 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9114e4-6fc5-410b-a347-362d577843ba None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:11.692532 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9114e4-6fc5-410b-a347-362d577843ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:11.761157 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9114e4-6fc5-410b-a347-362d577843ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:11.762544 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9114e4-6fc5-410b-a347-362d577843ba None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:11.849968 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:11.866905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 100/203] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 13 12:57:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 13 12:57:11.872419 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9b523-7797-41ef-bc37-f8870b8afbb4 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:11.975395 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:11.986410 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9b523-7797-41ef-bc37-f8870b8afbb4 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:12.022247 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b094e-dab1-45d5-b038-b5288c835078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:12.023151 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b094e-dab1-45d5-b038-b5288c835078 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:12.023566 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b094e-dab1-45d5-b038-b5288c835078 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:12.023969 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b094e-dab1-45d5-b038-b5288c835078 None admin] PATH_INFO: `/v3/projects/49ac31512d2943f2a28c22af2965292f` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:12.025306 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b094e-dab1-45d5-b038-b5288c835078 None admin] RBAC: Authorizing `identity:get_project(project_id=49ac31512d2943f2a28c22af2965292f)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:12.026891 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b094e-dab1-45d5-b038-b5288c835078 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:12.034058 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 104/204] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 13 12:57:11 2022] GET /identity/v3/projects/49ac31512d2943f2a28c22af2965292f => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:12.040147 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9114e4-6fc5-410b-a347-362d577843ba None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:12.146135 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9114e4-6fc5-410b-a347-362d577843ba None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:12.181342 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a867851-cd8d-4006-877d-09d1025b0c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:12.182172 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a867851-cd8d-4006-877d-09d1025b0c5d None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:12.182663 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a867851-cd8d-4006-877d-09d1025b0c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:12.183144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a867851-cd8d-4006-877d-09d1025b0c5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:12.184063 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a867851-cd8d-4006-877d-09d1025b0c5d None admin] RBAC: Adding query filter params (role.id=10ba22bff4d24fd69e4c6def38370617, scope.project.id=49ac31512d2943f2a28c22af2965292f, user.id=a2622b0eb71446d88292fd57ae27fd0f) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:12.184567 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a867851-cd8d-4006-877d-09d1025b0c5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:12.186270 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a867851-cd8d-4006-877d-09d1025b0c5d None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:12.196246 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 101/205] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 13 12:57:12 2022] GET /identity/v3/role_assignments?user.id=a2622b0eb71446d88292fd57ae27fd0f&scope.project.id=49ac31512d2943f2a28c22af2965292f&role.id=10ba22bff4d24fd69e4c6def38370617 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:13.324301 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e430ea5-ec8d-413f-9e07-fad9e73a214d None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:13.324831 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e430ea5-ec8d-413f-9e07-fad9e73a214d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:13.324831 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e430ea5-ec8d-413f-9e07-fad9e73a214d None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:13.326896 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 105/206] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:13.331937 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d45615-d391-4aa4-b996-65811566856b None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:13.332332 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d45615-d391-4aa4-b996-65811566856b None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:13.332509 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d45615-d391-4aa4-b996-65811566856b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:13.340053 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb9e27-b07f-4bdc-905d-f4ead69e926a None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:13.340339 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb9e27-b07f-4bdc-905d-f4ead69e926a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:13.340613 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb9e27-b07f-4bdc-905d-f4ead69e926a None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:13.342493 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 106/207] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:13.347425 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b33d7c-88c2-49b5-b61e-2d8c3f073baf None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:13.347709 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b33d7c-88c2-49b5-b61e-2d8c3f073baf None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:13.347976 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b33d7c-88c2-49b5-b61e-2d8c3f073baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:13.352439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-b0d45615-d391-4aa4-b996-65811566856b None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:13.353637 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-b0d45615-d391-4aa4-b996-65811566856b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:13.365280 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-19b33d7c-88c2-49b5-b61e-2d8c3f073baf None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:13.366424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-19b33d7c-88c2-49b5-b61e-2d8c3f073baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:13.399491 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:13.419466 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 102/208] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:13.420726 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46d2a6-29fa-431b-9acc-cb9afc8e9a14 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:13.421007 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46d2a6-29fa-431b-9acc-cb9afc8e9a14 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:13.421273 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46d2a6-29fa-431b-9acc-cb9afc8e9a14 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:13.422676 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 103/209] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:13.427711 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de904f28-9f52-4cf1-af07-048f8fc65552 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:13.428000 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de904f28-9f52-4cf1-af07-048f8fc65552 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:13.428270 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de904f28-9f52-4cf1-af07-048f8fc65552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:13.433093 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 107/210] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 13 12:57:13.448549 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-de904f28-9f52-4cf1-af07-048f8fc65552 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:13.449720 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-de904f28-9f52-4cf1-af07-048f8fc65552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:13.514481 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 104/211] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 13 12:57:13.686787 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b33d7c-88c2-49b5-b61e-2d8c3f073baf None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:13.703174 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de904f28-9f52-4cf1-af07-048f8fc65552 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:13.784338 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:13.797137 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b33d7c-88c2-49b5-b61e-2d8c3f073baf None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:13.807597 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de904f28-9f52-4cf1-af07-048f8fc65552 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:13.818153 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:13.831597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2391b625-f590-4edf-9e46-8e0e1d4fdf67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:13.832214 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391b625-f590-4edf-9e46-8e0e1d4fdf67 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:13.832486 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391b625-f590-4edf-9e46-8e0e1d4fdf67 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:13.832752 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391b625-f590-4edf-9e46-8e0e1d4fdf67 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:13.842877 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2391b625-f590-4edf-9e46-8e0e1d4fdf67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:13.843571 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679103e3-cab0-474d-a24b-6bdd9da0f3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:13.844192 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679103e3-cab0-474d-a24b-6bdd9da0f3ce None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:13.844466 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679103e3-cab0-474d-a24b-6bdd9da0f3ce None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:13.844731 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679103e3-cab0-474d-a24b-6bdd9da0f3ce None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:13.847517 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-2391b625-f590-4edf-9e46-8e0e1d4fdf67 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 13 12:57:13.847517 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:13.847517 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:13.847517 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:13.847517 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:13.847517 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:13.847517 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:13.847517 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:13.847517 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:13.847517 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:13.847517 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:13.847517 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:13.847517 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:13.847517 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:13.847517 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:13.847517 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:13.848540 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:13.848540 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:13.848540 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:13.848540 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:13.848540 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:13.848540 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:13.848540 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:13.848540 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:13.848540 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:13.848540 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:13.848540 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:13.848540 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:13.848540 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:13.848540 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:13.848540 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:13.848540 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:13.849321 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:13.849321 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:13.849321 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:13.849321 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:13.849321 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:13.849321 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:13.849321 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:13.849321 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:13.849321 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:13.849321 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:13.849321 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 13 12:57:13.849321 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:13.849321 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 108/212] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:57:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:13.849321 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2391b625-f590-4edf-9e46-8e0e1d4fdf67 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:13.856564 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679103e3-cab0-474d-a24b-6bdd9da0f3ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:13.861687 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-679103e3-cab0-474d-a24b-6bdd9da0f3ce None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 13 12:57:13.861687 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:13.861687 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:13.861687 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:13.861687 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:13.861687 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:13.861687 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:13.861687 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:13.861687 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:13.861687 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:13.861687 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:13.861687 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:13.861687 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:13.861687 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:13.861687 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:13.861687 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:13.862761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:13.862761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:13.862761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:13.862761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:13.862761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:13.862761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:13.862761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:13.862761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:13.862761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:13.862761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:13.862761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:13.862761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:13.862761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:13.862761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:13.862761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:13.862761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:13.863693 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:13.863693 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:13.863693 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:13.863693 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:13.863693 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:13.863693 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:13.863693 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:13.863693 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:13.863693 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:13.863693 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:13.863693 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 13 12:57:13.863693 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:13.863693 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 105/213] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:57:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:57:13.863693 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679103e3-cab0-474d-a24b-6bdd9da0f3ce None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:13.953218 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2391b625-f590-4edf-9e46-8e0e1d4fdf67 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:13.964452 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679103e3-cab0-474d-a24b-6bdd9da0f3ce None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:13.985401 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c29b2-b4b9-4228-aca7-70ebb4b2c162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:13.986012 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c29b2-b4b9-4228-aca7-70ebb4b2c162 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:13.986287 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c29b2-b4b9-4228-aca7-70ebb4b2c162 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:13.986552 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c29b2-b4b9-4228-aca7-70ebb4b2c162 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:13.987644 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c29b2-b4b9-4228-aca7-70ebb4b2c162 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:13.989077 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c29b2-b4b9-4228-aca7-70ebb4b2c162 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:13.990612 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 109/214] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:13.991384 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c29b2-b4b9-4228-aca7-70ebb4b2c162 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:13.998244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540360b-d65e-46db-9dfd-e86cb042faaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:13.998844 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540360b-d65e-46db-9dfd-e86cb042faaf None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:13.999144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540360b-d65e-46db-9dfd-e86cb042faaf None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:13.999416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540360b-d65e-46db-9dfd-e86cb042faaf None admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:14.000066 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0540360b-d65e-46db-9dfd-e86cb042faaf None admin] RBAC: Adding query filter params (name=member) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:14.000368 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0540360b-d65e-46db-9dfd-e86cb042faaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:14.001723 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0540360b-d65e-46db-9dfd-e86cb042faaf None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:14.008409 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:14.009663 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 106/215] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 13 12:57:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:14.010397 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540360b-d65e-46db-9dfd-e86cb042faaf None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:14.041078 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:14.089908 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c29b2-b4b9-4228-aca7-70ebb4b2c162 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:14.107304 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540360b-d65e-46db-9dfd-e86cb042faaf None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:14.122357 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060499a1-9651-467a-9e6b-b46e751fed25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:14.123501 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060499a1-9651-467a-9e6b-b46e751fed25 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:14.123779 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060499a1-9651-467a-9e6b-b46e751fed25 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:14.124046 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060499a1-9651-467a-9e6b-b46e751fed25 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:14.124702 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060499a1-9651-467a-9e6b-b46e751fed25 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:14.124997 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060499a1-9651-467a-9e6b-b46e751fed25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:14.126349 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060499a1-9651-467a-9e6b-b46e751fed25 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:14.134135 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 110/216] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 13 12:57:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:14.134900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060499a1-9651-467a-9e6b-b46e751fed25 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:14.141584 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef7d8dd-7a39-4ca8-b5e5-67b32f245838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:14.142182 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef7d8dd-7a39-4ca8-b5e5-67b32f245838 None admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:14.142459 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef7d8dd-7a39-4ca8-b5e5-67b32f245838 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:14.142728 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef7d8dd-7a39-4ca8-b5e5-67b32f245838 None admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:14.143402 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef7d8dd-7a39-4ca8-b5e5-67b32f245838 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:14.144832 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef7d8dd-7a39-4ca8-b5e5-67b32f245838 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:14.165918 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 107/217] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 13 12:57:14 2022] POST /identity/v3/users => generated 293 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:57:14.166710 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef7d8dd-7a39-4ca8-b5e5-67b32f245838 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:14.221671 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:14.251332 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060499a1-9651-467a-9e6b-b46e751fed25 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:14.284428 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:14.294252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b2bc79-4a9f-4b99-9cb9-2d9a5361283b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:14.295085 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2bc79-4a9f-4b99-9cb9-2d9a5361283b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:14.295373 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2bc79-4a9f-4b99-9cb9-2d9a5361283b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:14.295652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2bc79-4a9f-4b99-9cb9-2d9a5361283b None admin] PATH_INFO: `/v3/users/f171a5b073874df89ed1e9f795828a9d` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:14.299580 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef7d8dd-7a39-4ca8-b5e5-67b32f245838 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:14.312286 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b2bc79-4a9f-4b99-9cb9-2d9a5361283b None admin] RBAC: Authorizing `identity:get_user(user_id=f171a5b073874df89ed1e9f795828a9d)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:14.314025 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b2bc79-4a9f-4b99-9cb9-2d9a5361283b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:14.320893 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 111/218] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:57:14 2022] GET /identity/v3/users/f171a5b073874df89ed1e9f795828a9d => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:14.321828 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b2bc79-4a9f-4b99-9cb9-2d9a5361283b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:14.341394 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db90677-2ac9-4c47-8800-db1bee518396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:14.342647 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db90677-2ac9-4c47-8800-db1bee518396 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:14.342927 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db90677-2ac9-4c47-8800-db1bee518396 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:14.343262 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db90677-2ac9-4c47-8800-db1bee518396 None admin] PATH_INFO: `/v3/users/1efc885e761842dfad53f92df5a4c4ba` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:14.344390 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db90677-2ac9-4c47-8800-db1bee518396 None admin] RBAC: Authorizing `identity:get_user(user_id=1efc885e761842dfad53f92df5a4c4ba)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:14.345855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db90677-2ac9-4c47-8800-db1bee518396 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:14.347681 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 108/219] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:57:14 2022] GET /identity/v3/users/1efc885e761842dfad53f92df5a4c4ba => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:14.353197 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db90677-2ac9-4c47-8800-db1bee518396 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:14.443597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b2bc79-4a9f-4b99-9cb9-2d9a5361283b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:14.473931 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db90677-2ac9-4c47-8800-db1bee518396 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:14.492744 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c93727-7e43-4af2-8ad3-72401b9426c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:14.493366 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c93727-7e43-4af2-8ad3-72401b9426c7 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:14.493648 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c93727-7e43-4af2-8ad3-72401b9426c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:14.493924 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c93727-7e43-4af2-8ad3-72401b9426c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:14.494657 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c93727-7e43-4af2-8ad3-72401b9426c7 None admin] RBAC: Adding query filter params (role.id=10ba22bff4d24fd69e4c6def38370617, user.id=f171a5b073874df89ed1e9f795828a9d, scope.system=all) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:14.494971 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c93727-7e43-4af2-8ad3-72401b9426c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:14.496756 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c93727-7e43-4af2-8ad3-72401b9426c7 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:14.500481 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:14.504634 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 112/220] 10.222.0.23 () {56 vars in 1308 bytes} [Thu Oct 13 12:57:14 2022] GET /identity/v3/role_assignments?user.id=f171a5b073874df89ed1e9f795828a9d&scope.system=all&role.id=10ba22bff4d24fd69e4c6def38370617 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:14.522745 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6970099-c12b-425f-99d3-c37ce5c0e5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:14.523431 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6970099-c12b-425f-99d3-c37ce5c0e5b0 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:14.523709 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6970099-c12b-425f-99d3-c37ce5c0e5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:14.523979 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6970099-c12b-425f-99d3-c37ce5c0e5b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:14.554771 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:14.604985 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6970099-c12b-425f-99d3-c37ce5c0e5b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:14.606471 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6970099-c12b-425f-99d3-c37ce5c0e5b0 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:14.719130 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 109/221] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 13 12:57:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 13 12:57:14.729152 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c93727-7e43-4af2-8ad3-72401b9426c7 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:14.833704 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c93727-7e43-4af2-8ad3-72401b9426c7 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:14.868578 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04915d46-5ec3-4806-b92f-c1feb9e9f112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:14.869444 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04915d46-5ec3-4806-b92f-c1feb9e9f112 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:14.869857 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04915d46-5ec3-4806-b92f-c1feb9e9f112 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:14.870260 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04915d46-5ec3-4806-b92f-c1feb9e9f112 None admin] PATH_INFO: `/v3/projects/49ac31512d2943f2a28c22af2965292f` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:14.871598 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04915d46-5ec3-4806-b92f-c1feb9e9f112 None admin] RBAC: Authorizing `identity:get_project(project_id=49ac31512d2943f2a28c22af2965292f)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:14.873752 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04915d46-5ec3-4806-b92f-c1feb9e9f112 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:14.875845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 113/222] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 13 12:57:14 2022] GET /identity/v3/projects/49ac31512d2943f2a28c22af2965292f => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:14.881961 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6970099-c12b-425f-99d3-c37ce5c0e5b0 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:14.959431 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:14.989681 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6970099-c12b-425f-99d3-c37ce5c0e5b0 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:15.025945 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dcf7b7-8601-43d4-9fac-5f5d8d863340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:15.026555 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcf7b7-8601-43d4-9fac-5f5d8d863340 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:15.026884 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcf7b7-8601-43d4-9fac-5f5d8d863340 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:15.027183 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcf7b7-8601-43d4-9fac-5f5d8d863340 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:15.027953 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dcf7b7-8601-43d4-9fac-5f5d8d863340 None admin] RBAC: Adding query filter params (role.id=4844dbf0f3e14acfaa7b2207a67377a7, scope.project.id=49ac31512d2943f2a28c22af2965292f, user.id=1efc885e761842dfad53f92df5a4c4ba) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:15.028263 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dcf7b7-8601-43d4-9fac-5f5d8d863340 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:15.029730 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dcf7b7-8601-43d4-9fac-5f5d8d863340 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:15.036150 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 110/223] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 13 12:57:14 2022] GET /identity/v3/role_assignments?user.id=1efc885e761842dfad53f92df5a4c4ba&scope.project.id=49ac31512d2943f2a28c22af2965292f&role.id=4844dbf0f3e14acfaa7b2207a67377a7 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:16.506252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd307f0-a660-40e9-a56f-9897579320ef None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:16.506792 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd307f0-a660-40e9-a56f-9897579320ef None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:16.506792 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd307f0-a660-40e9-a56f-9897579320ef None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:16.508210 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 114/224] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:16.513493 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf5294-8d39-44eb-8ced-be7152970a3d None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:16.513870 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf5294-8d39-44eb-8ced-be7152970a3d None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:16.514031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf5294-8d39-44eb-8ced-be7152970a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:16.516482 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb58f5d-4c62-4f4d-9ddb-bf50cfea9ab4 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:16.516766 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb58f5d-4c62-4f4d-9ddb-bf50cfea9ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:16.517037 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb58f5d-4c62-4f4d-9ddb-bf50cfea9ab4 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:16.518304 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 115/225] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 13 12:57:16.523683 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cd5c7-0d21-4d5e-8443-8fd9f6f53645 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:16.523960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cd5c7-0d21-4d5e-8443-8fd9f6f53645 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:16.524228 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cd5c7-0d21-4d5e-8443-8fd9f6f53645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:16.533301 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-abdf5294-8d39-44eb-8ced-be7152970a3d None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:16.534492 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-abdf5294-8d39-44eb-8ced-be7152970a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:16.543916 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-654cd5c7-0d21-4d5e-8443-8fd9f6f53645 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:16.545802 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-654cd5c7-0d21-4d5e-8443-8fd9f6f53645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:16.614368 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 111/226] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:16.618531 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:16.627182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 116/227] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:16.683975 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d9d0f8-09e5-4fa2-85b9-f3e5c7b59d85 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:16.684285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d9d0f8-09e5-4fa2-85b9-f3e5c7b59d85 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:16.684521 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d9d0f8-09e5-4fa2-85b9-f3e5c7b59d85 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:16.685911 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 112/228] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:16.691226 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd2a3a-f07b-4023-8554-5c41c4a26692 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:16.691519 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd2a3a-f07b-4023-8554-5c41c4a26692 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:16.691798 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd2a3a-f07b-4023-8554-5c41c4a26692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:16.710193 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-edfd2a3a-f07b-4023-8554-5c41c4a26692 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:16.711406 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-edfd2a3a-f07b-4023-8554-5c41c4a26692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:16.776864 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 117/229] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:16.884542 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d9d0f8-09e5-4fa2-85b9-f3e5c7b59d85 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:16.889478 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd2a3a-f07b-4023-8554-5c41c4a26692 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:16.949082 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:17.000596 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd2a3a-f07b-4023-8554-5c41c4a26692 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:17.003352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d9d0f8-09e5-4fa2-85b9-f3e5c7b59d85 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:17.026244 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:17.034784 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0da9db-61ad-4d51-a2b7-b9e483e16be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:17.035383 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0da9db-61ad-4d51-a2b7-b9e483e16be1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:17.035659 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0da9db-61ad-4d51-a2b7-b9e483e16be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:17.035933 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0da9db-61ad-4d51-a2b7-b9e483e16be1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:17.037022 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0da9db-61ad-4d51-a2b7-b9e483e16be1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:17.038503 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0da9db-61ad-4d51-a2b7-b9e483e16be1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:17.039179 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d24fe-4b76-4012-9007-e287f33c9db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:17.039816 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d24fe-4b76-4012-9007-e287f33c9db9 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:17.040056 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 118/230] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:57:17.043454 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d24fe-4b76-4012-9007-e287f33c9db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:17.043454 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d24fe-4b76-4012-9007-e287f33c9db9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:17.044105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0da9db-61ad-4d51-a2b7-b9e483e16be1 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:17.052573 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d24fe-4b76-4012-9007-e287f33c9db9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:17.057452 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-d70d24fe-4b76-4012-9007-e287f33c9db9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 13 12:57:17.057452 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:17.057452 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:17.057452 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:17.057452 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:17.057452 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:17.057452 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:17.057452 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:17.057452 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:17.057452 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:17.057452 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:17.057452 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:17.057452 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:17.057452 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:17.057452 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:17.057452 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:17.058324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:17.058324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:17.058324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:17.058324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:17.058324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:17.058324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:17.058324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:17.058324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:17.058324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:17.058324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:17.058324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:17.058324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:17.058324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:17.058324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:17.058324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:17.058324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:17.059126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:17.059126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:17.059126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:17.059126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:17.059126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:17.059126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:17.059126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:17.059126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:17.059126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:17.059126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:17.059126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 13 12:57:17.059126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:17.059126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 113/231] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:57:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:17.059812 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d24fe-4b76-4012-9007-e287f33c9db9 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:17.144919 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0da9db-61ad-4d51-a2b7-b9e483e16be1 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:17.159551 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d24fe-4b76-4012-9007-e287f33c9db9 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:17.169896 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:17.176972 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b4a098-e4b1-44c2-962f-2e607e0aa398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:17.177594 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b4a098-e4b1-44c2-962f-2e607e0aa398 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:17.177872 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b4a098-e4b1-44c2-962f-2e607e0aa398 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:17.178140 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b4a098-e4b1-44c2-962f-2e607e0aa398 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:17.188689 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b4a098-e4b1-44c2-962f-2e607e0aa398 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:17.193171 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c8730-282c-41c8-978f-299c750fdd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:17.193783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-c0b4a098-e4b1-44c2-962f-2e607e0aa398 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 13 12:57:17.193783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:17.193783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:17.193783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:17.193783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:17.193783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:17.193783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:17.193783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:17.193783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:17.193783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:17.193783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:17.193783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:17.193783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:17.193783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:17.193783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:17.193783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:17.194598 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:17.194598 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:17.194598 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:17.194598 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:17.194598 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:17.194598 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:17.194598 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:17.194598 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:17.194598 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:17.194598 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:17.194598 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:17.194598 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:17.194598 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:17.194598 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:17.194598 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:17.194598 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:17.195637 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:17.195637 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:17.195637 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:17.195637 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:17.195637 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:17.195637 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:17.195637 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:17.195637 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:17.195637 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:17.195637 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:17.195637 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 13 12:57:17.195637 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:17.196494 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c8730-282c-41c8-978f-299c750fdd46 None admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:17.196494 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c8730-282c-41c8-978f-299c750fdd46 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:17.196494 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c8730-282c-41c8-978f-299c750fdd46 None admin] PATH_INFO: `/v3/groups` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:17.196679 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 119/232] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:57:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:57:17.196679 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b4a098-e4b1-44c2-962f-2e607e0aa398 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:17.196795 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c8730-282c-41c8-978f-299c750fdd46 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:17.197573 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c8730-282c-41c8-978f-299c750fdd46 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:17.213205 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 114/233] 10.222.0.23 () {60 vars in 1134 bytes} [Thu Oct 13 12:57:17 2022] POST /identity/v3/groups => generated 229 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:57:17.214005 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c8730-282c-41c8-978f-299c750fdd46 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:17.259077 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:17.313952 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b4a098-e4b1-44c2-962f-2e607e0aa398 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:17.327624 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c8730-282c-41c8-978f-299c750fdd46 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:17.355336 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1b35e-8da0-45e5-afd4-d32e9b78287b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:17.356306 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1b35e-8da0-45e5-afd4-d32e9b78287b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:17.356770 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1b35e-8da0-45e5-afd4-d32e9b78287b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:17.357153 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1b35e-8da0-45e5-afd4-d32e9b78287b None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:17.357852 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1b35e-8da0-45e5-afd4-d32e9b78287b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:17.358334 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1b35e-8da0-45e5-afd4-d32e9b78287b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:17.360523 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1b35e-8da0-45e5-afd4-d32e9b78287b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:17.363585 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322aa4a-00d6-41ec-9b21-653fdabe39c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:17.364212 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322aa4a-00d6-41ec-9b21-653fdabe39c6 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:17.364489 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322aa4a-00d6-41ec-9b21-653fdabe39c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:17.364760 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322aa4a-00d6-41ec-9b21-653fdabe39c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:17.365432 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7322aa4a-00d6-41ec-9b21-653fdabe39c6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:17.365737 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7322aa4a-00d6-41ec-9b21-653fdabe39c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:17.367155 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7322aa4a-00d6-41ec-9b21-653fdabe39c6 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:17.376679 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 120/234] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 13 12:57:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:17.377604 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1b35e-8da0-45e5-afd4-d32e9b78287b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:17.381546 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 115/235] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 13 12:57:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:17.382588 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322aa4a-00d6-41ec-9b21-653fdabe39c6 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:17.420406 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:17.488472 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1b35e-8da0-45e5-afd4-d32e9b78287b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:17.492851 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322aa4a-00d6-41ec-9b21-653fdabe39c6 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:17.506894 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:17.525597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456d196-a5b2-4fed-ab69-29aaf81e6f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:17.526231 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456d196-a5b2-4fed-ab69-29aaf81e6f8b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:17.526513 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456d196-a5b2-4fed-ab69-29aaf81e6f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:17.526786 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456d196-a5b2-4fed-ab69-29aaf81e6f8b None admin] PATH_INFO: `/v3/users/c0779e424b184b2994c8ee58c5a24706` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:17.536163 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fba5e0-c9ac-468c-8cc7-eaa75a06d45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:17.536811 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fba5e0-c9ac-468c-8cc7-eaa75a06d45b None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:17.537069 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fba5e0-c9ac-468c-8cc7-eaa75a06d45b None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:17.537340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fba5e0-c9ac-468c-8cc7-eaa75a06d45b None admin] PATH_INFO: `/v3/users/f171a5b073874df89ed1e9f795828a9d` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:17.538481 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fba5e0-c9ac-468c-8cc7-eaa75a06d45b None admin] RBAC: Authorizing `identity:get_user(user_id=f171a5b073874df89ed1e9f795828a9d)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:17.540071 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fba5e0-c9ac-468c-8cc7-eaa75a06d45b None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:17.541105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456d196-a5b2-4fed-ab69-29aaf81e6f8b None admin] RBAC: Authorizing `identity:get_user(user_id=c0779e424b184b2994c8ee58c5a24706)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:17.546122 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 116/236] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:57:17 2022] GET /identity/v3/users/f171a5b073874df89ed1e9f795828a9d => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:17.546250 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456d196-a5b2-4fed-ab69-29aaf81e6f8b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:17.546250 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 121/237] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:57:17 2022] GET /identity/v3/users/c0779e424b184b2994c8ee58c5a24706 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:57:17.547398 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fba5e0-c9ac-468c-8cc7-eaa75a06d45b None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:17.550007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456d196-a5b2-4fed-ab69-29aaf81e6f8b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:17.647363 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:17.650862 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456d196-a5b2-4fed-ab69-29aaf81e6f8b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:17.651254 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fba5e0-c9ac-468c-8cc7-eaa75a06d45b None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:17.685376 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0eb49d-7cfb-4930-8ab5-076bf3cef1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:17.685933 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0eb49d-7cfb-4930-8ab5-076bf3cef1df None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:17.686265 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0eb49d-7cfb-4930-8ab5-076bf3cef1df None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:17.686475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0eb49d-7cfb-4930-8ab5-076bf3cef1df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:17.687218 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0eb49d-7cfb-4930-8ab5-076bf3cef1df None admin] RBAC: Adding query filter params (role.id=4844dbf0f3e14acfaa7b2207a67377a7, user.id=c0779e424b184b2994c8ee58c5a24706, scope.system=all) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:17.687786 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0eb49d-7cfb-4930-8ab5-076bf3cef1df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:17.688828 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5aaec3-9e45-4e4f-bbbf-38f77fef8ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:17.689205 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0eb49d-7cfb-4930-8ab5-076bf3cef1df None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:17.689478 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5aaec3-9e45-4e4f-bbbf-38f77fef8ef2 None admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:17.689752 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5aaec3-9e45-4e4f-bbbf-38f77fef8ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:17.690021 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5aaec3-9e45-4e4f-bbbf-38f77fef8ef2 None admin] PATH_INFO: `/v3/system/users/f171a5b073874df89ed1e9f795828a9d/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:17.691453 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5aaec3-9e45-4e4f-bbbf-38f77fef8ef2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f171a5b073874df89ed1e9f795828a9d, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:17.692811 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5aaec3-9e45-4e4f-bbbf-38f77fef8ef2 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:17.694529 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 122/238] 10.222.0.23 () {56 vars in 1308 bytes} [Thu Oct 13 12:57:17 2022] GET /identity/v3/role_assignments?user.id=c0779e424b184b2994c8ee58c5a24706&scope.system=all&role.id=4844dbf0f3e14acfaa7b2207a67377a7 => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:57:17.705922 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 117/239] 10.222.0.23 () {58 vars in 1334 bytes} [Thu Oct 13 12:57:17 2022] PUT /identity/v3/system/users/f171a5b073874df89ed1e9f795828a9d/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 12:57:19.409586 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c14db7-2c05-456d-8d58-04a558fa0373 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:19.410098 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c14db7-2c05-456d-8d58-04a558fa0373 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:19.410170 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c14db7-2c05-456d-8d58-04a558fa0373 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:19.411542 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 123/240] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:19.416830 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5f8f7-9867-4fc7-84fb-690d8a546aff None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:19.417222 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5f8f7-9867-4fc7-84fb-690d8a546aff None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:19.417396 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5f8f7-9867-4fc7-84fb-690d8a546aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:19.435824 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-1cf5f8f7-9867-4fc7-84fb-690d8a546aff None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:19.436997 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-1cf5f8f7-9867-4fc7-84fb-690d8a546aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:19.505648 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 118/241] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:19.777197 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c14db7-2c05-456d-8d58-04a558fa0373 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:19.821481 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:19.848764 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecf3cc-7d57-44d7-b86b-2a20028ff949 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:19.849273 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecf3cc-7d57-44d7-b86b-2a20028ff949 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:19.849542 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecf3cc-7d57-44d7-b86b-2a20028ff949 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:19.850885 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 119/242] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 13 12:57:19.851919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4823c97-7ac2-48d4-8020-73b1d8aa8547 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:19.852197 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4823c97-7ac2-48d4-8020-73b1d8aa8547 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:19.852462 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4823c97-7ac2-48d4-8020-73b1d8aa8547 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:19.853836 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 120/243] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:19.857150 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77162ec2-1ce8-4249-81f2-822d89771baf None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:19.857515 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77162ec2-1ce8-4249-81f2-822d89771baf None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:19.857692 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77162ec2-1ce8-4249-81f2-822d89771baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:19.875446 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-77162ec2-1ce8-4249-81f2-822d89771baf None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:19.876607 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-77162ec2-1ce8-4249-81f2-822d89771baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:19.881244 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c14db7-2c05-456d-8d58-04a558fa0373 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:19.914898 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ef28c-258e-44cb-b94e-9aa1bf73fe5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:19.915550 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ef28c-258e-44cb-b94e-9aa1bf73fe5b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:19.915830 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ef28c-258e-44cb-b94e-9aa1bf73fe5b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:19.916099 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ef28c-258e-44cb-b94e-9aa1bf73fe5b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:19.917175 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743ef28c-258e-44cb-b94e-9aa1bf73fe5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:19.918610 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743ef28c-258e-44cb-b94e-9aa1bf73fe5b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:19.920218 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 124/244] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:19.921303 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f7ee6-5eba-4cd2-a92f-62fbc5230dfb None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:19.921728 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f7ee6-5eba-4cd2-a92f-62fbc5230dfb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:19.922003 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f7ee6-5eba-4cd2-a92f-62fbc5230dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:19.939314 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-fc1f7ee6-5eba-4cd2-a92f-62fbc5230dfb None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:19.940479 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fc1f7ee6-5eba-4cd2-a92f-62fbc5230dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:19.948486 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 121/245] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 13 12:57:19.949304 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77162ec2-1ce8-4249-81f2-822d89771baf None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:19.978362 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:20.014408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 125/246] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:20.057049 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77162ec2-1ce8-4249-81f2-822d89771baf None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:20.090873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c33814-0d33-47da-8b5f-0687800e739a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:20.092057 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c33814-0d33-47da-8b5f-0687800e739a None admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:20.092334 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c33814-0d33-47da-8b5f-0687800e739a None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:20.092626 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c33814-0d33-47da-8b5f-0687800e739a None admin] PATH_INFO: `/v3/groups` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:20.093252 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c33814-0d33-47da-8b5f-0687800e739a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:20.094682 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c33814-0d33-47da-8b5f-0687800e739a None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:20.108968 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 122/247] 10.222.0.23 () {60 vars in 1134 bytes} [Thu Oct 13 12:57:19 2022] POST /identity/v3/groups => generated 226 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:57:20.214686 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f7ee6-5eba-4cd2-a92f-62fbc5230dfb None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:20.257037 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:20.282428 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c33814-0d33-47da-8b5f-0687800e739a None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:20.324638 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f7ee6-5eba-4cd2-a92f-62fbc5230dfb None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:20.363823 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb3243-4579-4532-aec8-3117fac741b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:20.364459 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb3243-4579-4532-aec8-3117fac741b3 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:20.364735 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb3243-4579-4532-aec8-3117fac741b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:20.365004 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb3243-4579-4532-aec8-3117fac741b3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:20.379742 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb3243-4579-4532-aec8-3117fac741b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:20.385673 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-2fdb3243-4579-4532-aec8-3117fac741b3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 13 12:57:20.385673 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:20.385673 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:20.385673 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:20.385673 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:20.385673 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:20.385673 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:20.385673 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:20.385673 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:20.385673 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:20.385673 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:20.385673 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:20.385673 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:20.385673 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:20.385673 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:20.385673 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:20.386553 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:20.386553 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:20.386553 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:20.386553 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:20.386553 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:20.386553 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:20.386553 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:20.386553 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:20.386553 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:20.386553 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:20.386553 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:20.386553 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:20.386553 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:20.386553 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:20.386553 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:20.386553 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:20.387529 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:20.387529 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:20.387529 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:20.387529 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:20.387529 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:20.387529 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:20.387529 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:20.387529 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:20.387529 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:20.387529 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:20.387529 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 13 12:57:20.387529 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:20.387529 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 126/248] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:57:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:20.392662 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb3243-4579-4532-aec8-3117fac741b3 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:20.400114 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:20.420687 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c33814-0d33-47da-8b5f-0687800e739a None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:20.466943 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4982687-f67e-487a-a3cf-1caac7f42519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:20.467626 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4982687-f67e-487a-a3cf-1caac7f42519 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:20.467909 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4982687-f67e-487a-a3cf-1caac7f42519 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:20.468425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4982687-f67e-487a-a3cf-1caac7f42519 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:20.481212 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4982687-f67e-487a-a3cf-1caac7f42519 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:20.486347 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-d4982687-f67e-487a-a3cf-1caac7f42519 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 13 12:57:20.486347 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:20.486347 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:20.486347 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:20.486347 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:20.486347 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:20.486347 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:20.486347 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:20.486347 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:20.486347 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:20.486347 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:20.486347 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:20.486347 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:20.486347 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:20.486347 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:20.486347 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:20.487273 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:20.487273 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:20.487273 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:20.487273 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:20.487273 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:20.487273 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:20.487273 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:20.487273 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:20.487273 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:20.487273 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:20.487273 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:20.487273 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:20.487273 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:20.487273 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:20.487273 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:20.487273 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:20.488073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:20.488073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:20.488073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:20.488073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:20.488073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:20.488073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:20.488073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:20.488073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:20.488073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:20.488073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:20.488073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 13 12:57:20.488073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:20.488657 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 123/249] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:57:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:57:20.495480 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4982687-f67e-487a-a3cf-1caac7f42519 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:20.514501 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:20.519013 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb3243-4579-4532-aec8-3117fac741b3 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:20.574311 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a430b-ef00-4aa3-9be1-7074e59242a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:20.574969 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a430b-ef00-4aa3-9be1-7074e59242a7 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:20.575233 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a430b-ef00-4aa3-9be1-7074e59242a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:20.576053 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a430b-ef00-4aa3-9be1-7074e59242a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:20.576729 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a430b-ef00-4aa3-9be1-7074e59242a7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:20.577036 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a430b-ef00-4aa3-9be1-7074e59242a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:20.579154 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a430b-ef00-4aa3-9be1-7074e59242a7 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:20.590799 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 127/250] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 13 12:57:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:57:20.598083 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a430b-ef00-4aa3-9be1-7074e59242a7 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:20.635450 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4982687-f67e-487a-a3cf-1caac7f42519 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:20.677935 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deeb163-1da0-4ca8-a895-c5cb338be525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:20.678556 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deeb163-1da0-4ca8-a895-c5cb338be525 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:20.678834 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deeb163-1da0-4ca8-a895-c5cb338be525 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:20.679946 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deeb163-1da0-4ca8-a895-c5cb338be525 None admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:20.680665 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deeb163-1da0-4ca8-a895-c5cb338be525 None admin] RBAC: Adding query filter params (name=member) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:20.680925 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deeb163-1da0-4ca8-a895-c5cb338be525 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:20.682322 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deeb163-1da0-4ca8-a895-c5cb338be525 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:20.698055 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 124/251] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 13 12:57:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:20.704263 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deeb163-1da0-4ca8-a895-c5cb338be525 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:20.716330 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:20.726334 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a430b-ef00-4aa3-9be1-7074e59242a7 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:20.774594 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33374e1f-2373-470d-ad46-e407d12f4f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:20.775287 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33374e1f-2373-470d-ad46-e407d12f4f21 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:20.775573 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33374e1f-2373-470d-ad46-e407d12f4f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:20.776072 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33374e1f-2373-470d-ad46-e407d12f4f21 None admin] PATH_INFO: `/v3/users/c0779e424b184b2994c8ee58c5a24706` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:20.778392 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33374e1f-2373-470d-ad46-e407d12f4f21 None admin] RBAC: Authorizing `identity:get_user(user_id=c0779e424b184b2994c8ee58c5a24706)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:20.780443 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33374e1f-2373-470d-ad46-e407d12f4f21 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:20.782224 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 128/252] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:57:20 2022] GET /identity/v3/users/c0779e424b184b2994c8ee58c5a24706 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:20.789294 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33374e1f-2373-470d-ad46-e407d12f4f21 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:20.829369 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:20.852938 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deeb163-1da0-4ca8-a895-c5cb338be525 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:20.899260 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69538bbf-0db5-4882-a0d0-5f8205a2f242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:20.900283 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69538bbf-0db5-4882-a0d0-5f8205a2f242 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:20.900774 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69538bbf-0db5-4882-a0d0-5f8205a2f242 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:20.901232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69538bbf-0db5-4882-a0d0-5f8205a2f242 None admin] PATH_INFO: `/v3/users/f171a5b073874df89ed1e9f795828a9d` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:20.903399 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69538bbf-0db5-4882-a0d0-5f8205a2f242 None admin] RBAC: Authorizing `identity:get_user(user_id=f171a5b073874df89ed1e9f795828a9d)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:20.905368 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69538bbf-0db5-4882-a0d0-5f8205a2f242 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:20.908262 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 125/253] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:57:20 2022] GET /identity/v3/users/f171a5b073874df89ed1e9f795828a9d => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:57:20.919203 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69538bbf-0db5-4882-a0d0-5f8205a2f242 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:20.921749 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33374e1f-2373-470d-ad46-e407d12f4f21 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:20.966625 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceebc38-4b16-4131-9d99-8d3b5e112924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:20.967317 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceebc38-4b16-4131-9d99-8d3b5e112924 None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:20.967597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceebc38-4b16-4131-9d99-8d3b5e112924 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:20.967866 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceebc38-4b16-4131-9d99-8d3b5e112924 None admin] PATH_INFO: `/v3/system/users/c0779e424b184b2994c8ee58c5a24706/roles/4844dbf0f3e14acfaa7b2207a67377a7` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:20.969302 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceebc38-4b16-4131-9d99-8d3b5e112924 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c0779e424b184b2994c8ee58c5a24706, role_id=4844dbf0f3e14acfaa7b2207a67377a7)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:20.970676 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceebc38-4b16-4131-9d99-8d3b5e112924 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:20.983358 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 129/254] 10.222.0.23 () {58 vars in 1334 bytes} [Thu Oct 13 12:57:20 2022] PUT /identity/v3/system/users/c0779e424b184b2994c8ee58c5a24706/roles/4844dbf0f3e14acfaa7b2207a67377a7 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 12:57:21.054730 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:21.098254 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69538bbf-0db5-4882-a0d0-5f8205a2f242 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:21.145461 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd03a82e-f151-40f4-8529-23d16b691255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:21.146085 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03a82e-f151-40f4-8529-23d16b691255 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:21.146362 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03a82e-f151-40f4-8529-23d16b691255 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:21.146629 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03a82e-f151-40f4-8529-23d16b691255 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:21.147442 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd03a82e-f151-40f4-8529-23d16b691255 None admin] RBAC: Adding query filter params (role.id=10ba22bff4d24fd69e4c6def38370617, user.id=f171a5b073874df89ed1e9f795828a9d, scope.system=all) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:21.147810 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd03a82e-f151-40f4-8529-23d16b691255 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:21.149271 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd03a82e-f151-40f4-8529-23d16b691255 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:21.167123 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 126/255] 10.222.0.23 () {56 vars in 1308 bytes} [Thu Oct 13 12:57:20 2022] GET /identity/v3/role_assignments?user.id=f171a5b073874df89ed1e9f795828a9d&scope.system=all&role.id=10ba22bff4d24fd69e4c6def38370617 => generated 510 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:22.654542 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5938aa-c32f-42fe-92b8-2e3edff980c2 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:22.655079 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5938aa-c32f-42fe-92b8-2e3edff980c2 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:22.655187 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5938aa-c32f-42fe-92b8-2e3edff980c2 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:22.656691 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 130/256] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:22.662156 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84524d-3f83-4691-853d-56e1737200ab None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:22.662574 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84524d-3f83-4691-853d-56e1737200ab None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:22.662744 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84524d-3f83-4691-853d-56e1737200ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:22.688417 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-1c84524d-3f83-4691-853d-56e1737200ab None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:22.689694 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-1c84524d-3f83-4691-853d-56e1737200ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:22.767755 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 127/257] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 13 12:57:23.254763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f2d4b-b54b-48d4-b451-1cdef57eff78 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:23.255129 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f2d4b-b54b-48d4-b451-1cdef57eff78 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:23.255321 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f2d4b-b54b-48d4-b451-1cdef57eff78 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:23.256893 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 131/258] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:23.262276 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572dbda-ade4-4d1c-a668-8cc6db5b5789 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:23.262563 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572dbda-ade4-4d1c-a668-8cc6db5b5789 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:23.262894 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572dbda-ade4-4d1c-a668-8cc6db5b5789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:23.292541 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f2d4b-b54b-48d4-b451-1cdef57eff78 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:23.294307 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-4572dbda-ade4-4d1c-a668-8cc6db5b5789 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:23.295269 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-4572dbda-ade4-4d1c-a668-8cc6db5b5789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:23.305007 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 128/259] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 13 12:57:23.312979 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f2d4b-b54b-48d4-b451-1cdef57eff78 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:23.322324 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a7485-e7b5-4125-8c89-85a4d26b45d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:23.323241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a7485-e7b5-4125-8c89-85a4d26b45d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:23.323727 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a7485-e7b5-4125-8c89-85a4d26b45d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:23.324181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a7485-e7b5-4125-8c89-85a4d26b45d9 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:23.347158 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a7485-e7b5-4125-8c89-85a4d26b45d9 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:23.352285 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-577a7485-e7b5-4125-8c89-85a4d26b45d9 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 13 12:57:23.352285 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:23.352285 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:23.352285 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:23.352285 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:23.352285 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:23.352285 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:23.352285 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:23.352285 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:23.352285 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:23.352285 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:23.352285 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:23.352285 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:23.352285 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:23.352285 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:23.352285 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:23.353123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:23.353123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:23.353123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:23.353123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:23.353123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:23.353123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:23.353123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:23.353123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:23.353123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:23.353123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:23.353123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:23.353123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:23.353123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:23.353123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:23.353123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:23.353123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:23.353900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:23.353900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:23.353900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:23.353900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:23.353900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:23.353900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:23.353900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:23.353900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:23.353900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:23.353900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:23.353900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 13 12:57:23.353900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:23.356210 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 132/260] 10.222.0.23 () {56 vars in 1120 bytes} [Thu Oct 13 12:57:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 64 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:57:23.362066 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572dbda-ade4-4d1c-a668-8cc6db5b5789 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:23.381447 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572dbda-ade4-4d1c-a668-8cc6db5b5789 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:23.387560 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df81d7d-91ff-461a-9a56-e01cf078aa53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:23.388231 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df81d7d-91ff-461a-9a56-e01cf078aa53 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:23.388509 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df81d7d-91ff-461a-9a56-e01cf078aa53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:23.388780 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df81d7d-91ff-461a-9a56-e01cf078aa53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:23.389453 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df81d7d-91ff-461a-9a56-e01cf078aa53 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:23.389752 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df81d7d-91ff-461a-9a56-e01cf078aa53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:23.391554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: WARNING py.warnings [None req-6df81d7d-91ff-461a-9a56-e01cf078aa53 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 13 12:57:23.391554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: warnings.warn(msg) Oct 13 12:57:23.391554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: Oct 13 12:57:23.392461 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df81d7d-91ff-461a-9a56-e01cf078aa53 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:23.407370 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 129/261] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Oct 13 12:57:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:23.412905 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a7485-e7b5-4125-8c89-85a4d26b45d9 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:23.425210 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a7485-e7b5-4125-8c89-85a4d26b45d9 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:23.430913 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89138701-006d-41d2-9470-a85bdfef0dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:23.431804 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89138701-006d-41d2-9470-a85bdfef0dbd admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:23.432294 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89138701-006d-41d2-9470-a85bdfef0dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:23.432761 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89138701-006d-41d2-9470-a85bdfef0dbd admin admin] PATH_INFO: `/v3/groups/1969cf5ef4e344b1b4128ef7e86df9ad` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:23.445166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89138701-006d-41d2-9470-a85bdfef0dbd admin admin] RBAC: Authorizing `identity:get_group(group_id=1969cf5ef4e344b1b4128ef7e86df9ad)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:23.447284 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: WARNING py.warnings [None req-89138701-006d-41d2-9470-a85bdfef0dbd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 13 12:57:23.447284 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: warnings.warn(msg) Oct 13 12:57:23.447284 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: Oct 13 12:57:23.448856 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89138701-006d-41d2-9470-a85bdfef0dbd admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:23.451615 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 133/262] 10.222.0.23 () {56 vars in 1201 bytes} [Thu Oct 13 12:57:23 2022] GET /identity/v3/groups/1969cf5ef4e344b1b4128ef7e86df9ad => generated 226 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:23.457125 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df81d7d-91ff-461a-9a56-e01cf078aa53 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:23.473241 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df81d7d-91ff-461a-9a56-e01cf078aa53 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:23.479319 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f357f2b-421b-4247-91ae-25df9ab44bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:23.479981 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f357f2b-421b-4247-91ae-25df9ab44bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:23.480263 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f357f2b-421b-4247-91ae-25df9ab44bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:23.480531 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f357f2b-421b-4247-91ae-25df9ab44bb1 admin admin] PATH_INFO: `/v3/projects/b14cb10007d440b4828ccc9869dc8aa2` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:23.482290 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f357f2b-421b-4247-91ae-25df9ab44bb1 admin admin] RBAC: Authorizing `identity:get_project(project_id=b14cb10007d440b4828ccc9869dc8aa2)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:23.484427 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f357f2b-421b-4247-91ae-25df9ab44bb1 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:23.486436 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 130/263] 10.222.0.23 () {56 vars in 1207 bytes} [Thu Oct 13 12:57:23 2022] GET /identity/v3/projects/b14cb10007d440b4828ccc9869dc8aa2 => generated 298 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:23.492223 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89138701-006d-41d2-9470-a85bdfef0dbd admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:23.508232 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89138701-006d-41d2-9470-a85bdfef0dbd admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:23.519034 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e91a53-8deb-4aa7-955c-a845e5a2a77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:23.519982 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e91a53-8deb-4aa7-955c-a845e5a2a77d admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:23.520517 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e91a53-8deb-4aa7-955c-a845e5a2a77d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:23.520979 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e91a53-8deb-4aa7-955c-a845e5a2a77d admin admin] PATH_INFO: `/v3/projects/b14cb10007d440b4828ccc9869dc8aa2/groups/1969cf5ef4e344b1b4128ef7e86df9ad/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:23.523073 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e91a53-8deb-4aa7-955c-a845e5a2a77d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b14cb10007d440b4828ccc9869dc8aa2, group_id=1969cf5ef4e344b1b4128ef7e86df9ad, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:23.524284 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: WARNING py.warnings [None req-36e91a53-8deb-4aa7-955c-a845e5a2a77d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 13 12:57:23.524284 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: warnings.warn(msg) Oct 13 12:57:23.524284 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: Oct 13 12:57:23.526593 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e91a53-8deb-4aa7-955c-a845e5a2a77d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:23.551291 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 134/264] 10.222.0.23 () {58 vars in 1463 bytes} [Thu Oct 13 12:57:23 2022] PUT /identity/v3/projects/b14cb10007d440b4828ccc9869dc8aa2/groups/1969cf5ef4e344b1b4128ef7e86df9ad/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 12:57:23.721115 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba8111-8d4b-4d78-b504-ddb360d96ee6 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:23.721617 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba8111-8d4b-4d78-b504-ddb360d96ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:23.721720 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba8111-8d4b-4d78-b504-ddb360d96ee6 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:23.723288 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 131/265] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:23.728526 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d30463-5c27-47fb-a0c7-add6442f7209 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:23.729338 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d30463-5c27-47fb-a0c7-add6442f7209 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:23.729804 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d30463-5c27-47fb-a0c7-add6442f7209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:23.755690 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-68d30463-5c27-47fb-a0c7-add6442f7209 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:23.757213 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-68d30463-5c27-47fb-a0c7-add6442f7209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:23.796296 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba8111-8d4b-4d78-b504-ddb360d96ee6 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:23.861730 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 135/266] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 13 12:57:23.904797 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba8111-8d4b-4d78-b504-ddb360d96ee6 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:23.909386 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79936357-5262-4858-a753-fe80566ad6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:23.910315 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79936357-5262-4858-a753-fe80566ad6bf admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:23.910392 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79936357-5262-4858-a753-fe80566ad6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:23.910612 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79936357-5262-4858-a753-fe80566ad6bf admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:23.921279 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:23.925678 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79936357-5262-4858-a753-fe80566ad6bf admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:23.930617 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-79936357-5262-4858-a753-fe80566ad6bf admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 13 12:57:23.930617 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:23.930617 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:23.930617 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:23.930617 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:23.930617 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:23.930617 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:23.930617 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:23.930617 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:23.930617 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:23.930617 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:23.930617 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:23.930617 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:23.930617 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:23.930617 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:23.930617 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:23.931507 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:23.931507 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:23.931507 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:23.931507 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:23.931507 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:23.931507 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:23.931507 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:23.931507 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:23.931507 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:23.931507 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:23.931507 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:23.931507 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:23.931507 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:23.931507 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:23.931507 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:23.931507 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:23.932278 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:23.932278 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:23.932278 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:23.932278 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:23.932278 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:23.932278 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:23.932278 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:23.932278 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:23.932278 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:23.932278 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:23.932278 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 13 12:57:23.932278 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:23.932278 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 132/267] 10.222.0.23 () {56 vars in 1135 bytes} [Thu Oct 13 12:57:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 137 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:57:23.939549 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d30463-5c27-47fb-a0c7-add6442f7209 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:23.956422 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d30463-5c27-47fb-a0c7-add6442f7209 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:23.967728 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934fdfa-78f0-4717-992e-8175fce0ab6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:23.968348 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934fdfa-78f0-4717-992e-8175fce0ab6a admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:23.968632 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934fdfa-78f0-4717-992e-8175fce0ab6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:23.968907 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934fdfa-78f0-4717-992e-8175fce0ab6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:23.969591 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934fdfa-78f0-4717-992e-8175fce0ab6a admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:23.969898 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934fdfa-78f0-4717-992e-8175fce0ab6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:23.970964 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: WARNING py.warnings [None req-a934fdfa-78f0-4717-992e-8175fce0ab6a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 13 12:57:23.970964 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: warnings.warn(msg) Oct 13 12:57:23.970964 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: Oct 13 12:57:23.971959 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934fdfa-78f0-4717-992e-8175fce0ab6a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:23.973524 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:23.983319 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 136/268] 10.222.0.23 () {56 vars in 1132 bytes} [Thu Oct 13 12:57:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:23.989277 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79936357-5262-4858-a753-fe80566ad6bf admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:24.004274 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79936357-5262-4858-a753-fe80566ad6bf admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:24.012451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d47999-e3ee-4efd-9f28-be91ca994bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:24.013124 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d47999-e3ee-4efd-9f28-be91ca994bcd admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:24.013397 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d47999-e3ee-4efd-9f28-be91ca994bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:24.013664 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d47999-e3ee-4efd-9f28-be91ca994bcd admin admin] PATH_INFO: `/v3/groups/1969cf5ef4e344b1b4128ef7e86df9ad` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:24.014783 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d47999-e3ee-4efd-9f28-be91ca994bcd admin admin] RBAC: Authorizing `identity:get_group(group_id=1969cf5ef4e344b1b4128ef7e86df9ad)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:24.015863 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: WARNING py.warnings [None req-01d47999-e3ee-4efd-9f28-be91ca994bcd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 13 12:57:24.015863 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: warnings.warn(msg) Oct 13 12:57:24.015863 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: Oct 13 12:57:24.017073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d47999-e3ee-4efd-9f28-be91ca994bcd admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:24.018953 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 133/269] 10.222.0.23 () {56 vars in 1201 bytes} [Thu Oct 13 12:57:23 2022] GET /identity/v3/groups/1969cf5ef4e344b1b4128ef7e86df9ad => generated 226 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:24.024267 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934fdfa-78f0-4717-992e-8175fce0ab6a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:24.036217 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934fdfa-78f0-4717-992e-8175fce0ab6a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:24.041767 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddea448-4e5c-4b5f-8527-68d4d003a82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:24.042403 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddea448-4e5c-4b5f-8527-68d4d003a82a admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:24.042680 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddea448-4e5c-4b5f-8527-68d4d003a82a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:24.042952 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddea448-4e5c-4b5f-8527-68d4d003a82a admin admin] PATH_INFO: `/v3/projects/b14cb10007d440b4828ccc9869dc8aa2` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:24.044700 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddea448-4e5c-4b5f-8527-68d4d003a82a admin admin] RBAC: Authorizing `identity:get_project(project_id=b14cb10007d440b4828ccc9869dc8aa2)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:24.046816 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddea448-4e5c-4b5f-8527-68d4d003a82a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:24.048626 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 137/270] 10.222.0.23 () {56 vars in 1207 bytes} [Thu Oct 13 12:57:24 2022] GET /identity/v3/projects/b14cb10007d440b4828ccc9869dc8aa2 => generated 298 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:24.054216 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d47999-e3ee-4efd-9f28-be91ca994bcd admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:24.064916 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d47999-e3ee-4efd-9f28-be91ca994bcd admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:24.075178 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea40f5d-2d81-444a-8c68-1cc021aaac47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:24.075876 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea40f5d-2d81-444a-8c68-1cc021aaac47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:24.076161 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea40f5d-2d81-444a-8c68-1cc021aaac47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:24.076431 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea40f5d-2d81-444a-8c68-1cc021aaac47 admin admin] PATH_INFO: `/v3/projects/b14cb10007d440b4828ccc9869dc8aa2/groups/1969cf5ef4e344b1b4128ef7e86df9ad/roles/5a10c87b17414f04846918d777506811` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:24.078568 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea40f5d-2d81-444a-8c68-1cc021aaac47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b14cb10007d440b4828ccc9869dc8aa2, group_id=1969cf5ef4e344b1b4128ef7e86df9ad, role_id=5a10c87b17414f04846918d777506811)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:24.079633 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: WARNING py.warnings [None req-3ea40f5d-2d81-444a-8c68-1cc021aaac47 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 13 12:57:24.079633 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: warnings.warn(msg) Oct 13 12:57:24.079633 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: Oct 13 12:57:24.081604 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea40f5d-2d81-444a-8c68-1cc021aaac47 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:24.102795 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 134/271] 10.222.0.23 () {58 vars in 1463 bytes} [Thu Oct 13 12:57:24 2022] PUT /identity/v3/projects/b14cb10007d440b4828ccc9869dc8aa2/groups/1969cf5ef4e344b1b4128ef7e86df9ad/roles/5a10c87b17414f04846918d777506811 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 12:57:24.330405 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddea448-4e5c-4b5f-8527-68d4d003a82a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:24.348447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddea448-4e5c-4b5f-8527-68d4d003a82a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:24.415910 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e1052-664a-41a8-84c5-5de919f0e187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:24.416543 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e1052-664a-41a8-84c5-5de919f0e187 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:24.416826 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e1052-664a-41a8-84c5-5de919f0e187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:24.417092 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e1052-664a-41a8-84c5-5de919f0e187 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:24.430859 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296e1052-664a-41a8-84c5-5de919f0e187 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:24.436186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-296e1052-664a-41a8-84c5-5de919f0e187 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 13 12:57:24.436186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:24.436186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:24.436186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:24.436186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:24.436186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:24.436186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:24.436186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:24.436186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:24.436186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:24.436186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:24.436186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:24.436186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:24.436186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:24.436186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:24.436186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:24.438416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:24.438416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:24.438416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:24.438416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:24.438416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:24.438416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:24.438416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:24.438416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:24.438416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:24.438416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:24.438416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:24.438416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:24.438416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:24.438416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:24.438416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:24.438416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:24.440336 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:24.440336 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:24.440336 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:24.440336 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:24.440336 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:24.440336 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:24.440336 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:24.440336 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:24.440336 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:24.440336 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:24.440336 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 13 12:57:24.440336 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:24.440336 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 138/272] 10.222.0.23 () {56 vars in 1120 bytes} [Thu Oct 13 12:57:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 108 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:24.443383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea40f5d-2d81-444a-8c68-1cc021aaac47 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:24.456592 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea40f5d-2d81-444a-8c68-1cc021aaac47 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:24.464897 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f12e3-687b-4b6e-aa85-caa21a2afc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:24.465603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f12e3-687b-4b6e-aa85-caa21a2afc58 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:24.466456 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f12e3-687b-4b6e-aa85-caa21a2afc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:24.466734 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f12e3-687b-4b6e-aa85-caa21a2afc58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:24.467471 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f12e3-687b-4b6e-aa85-caa21a2afc58 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:24.467775 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f12e3-687b-4b6e-aa85-caa21a2afc58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:24.469443 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f12e3-687b-4b6e-aa85-caa21a2afc58 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:24.479246 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 135/273] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Oct 13 12:57:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:24.485422 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e1052-664a-41a8-84c5-5de919f0e187 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:24.500859 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e1052-664a-41a8-84c5-5de919f0e187 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:24.506125 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066fc159-f573-4b06-893b-35af992b6aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:24.506809 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066fc159-f573-4b06-893b-35af992b6aeb admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:24.507107 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066fc159-f573-4b06-893b-35af992b6aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:24.507381 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066fc159-f573-4b06-893b-35af992b6aeb admin admin] PATH_INFO: `/v3/groups/1969cf5ef4e344b1b4128ef7e86df9ad` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:24.508568 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066fc159-f573-4b06-893b-35af992b6aeb admin admin] RBAC: Authorizing `identity:get_group(group_id=1969cf5ef4e344b1b4128ef7e86df9ad)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:24.510551 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066fc159-f573-4b06-893b-35af992b6aeb admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:24.512166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 139/274] 10.222.0.23 () {56 vars in 1201 bytes} [Thu Oct 13 12:57:24 2022] GET /identity/v3/groups/1969cf5ef4e344b1b4128ef7e86df9ad => generated 226 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:24.518524 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f12e3-687b-4b6e-aa85-caa21a2afc58 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:24.529453 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f12e3-687b-4b6e-aa85-caa21a2afc58 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:24.539216 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed274e77-5c35-4456-89c8-674dfa431c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:24.539866 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed274e77-5c35-4456-89c8-674dfa431c61 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:24.540144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed274e77-5c35-4456-89c8-674dfa431c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:24.540417 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed274e77-5c35-4456-89c8-674dfa431c61 admin admin] PATH_INFO: `/v3/projects/49ac31512d2943f2a28c22af2965292f` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:24.541564 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed274e77-5c35-4456-89c8-674dfa431c61 admin admin] RBAC: Authorizing `identity:get_project(project_id=49ac31512d2943f2a28c22af2965292f)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:24.543650 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed274e77-5c35-4456-89c8-674dfa431c61 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:24.545311 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 136/275] 10.222.0.23 () {56 vars in 1207 bytes} [Thu Oct 13 12:57:24 2022] GET /identity/v3/projects/49ac31512d2943f2a28c22af2965292f => generated 302 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:24.551459 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066fc159-f573-4b06-893b-35af992b6aeb admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:24.564460 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066fc159-f573-4b06-893b-35af992b6aeb admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:24.568960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7ce5ec-1c91-4de2-a4fc-7547d7e69402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:24.569612 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ce5ec-1c91-4de2-a4fc-7547d7e69402 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:24.569890 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ce5ec-1c91-4de2-a4fc-7547d7e69402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:24.570160 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ce5ec-1c91-4de2-a4fc-7547d7e69402 admin admin] PATH_INFO: `/v3/projects/49ac31512d2943f2a28c22af2965292f/groups/1969cf5ef4e344b1b4128ef7e86df9ad/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:24.573468 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7ce5ec-1c91-4de2-a4fc-7547d7e69402 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49ac31512d2943f2a28c22af2965292f, group_id=1969cf5ef4e344b1b4128ef7e86df9ad, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:24.576863 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7ce5ec-1c91-4de2-a4fc-7547d7e69402 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:24.601564 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 140/276] 10.222.0.23 () {58 vars in 1463 bytes} [Thu Oct 13 12:57:24 2022] PUT /identity/v3/projects/49ac31512d2943f2a28c22af2965292f/groups/1969cf5ef4e344b1b4128ef7e86df9ad/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 12:57:24.624494 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c85916-4f1c-42fe-8e80-79c752b1ffde None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:24.624793 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c85916-4f1c-42fe-8e80-79c752b1ffde None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:24.625038 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c85916-4f1c-42fe-8e80-79c752b1ffde None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:24.626783 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 137/277] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:24.642907 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022d70a-6620-43c3-8319-eedd157d2251 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:24.642907 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022d70a-6620-43c3-8319-eedd157d2251 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:24.642907 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022d70a-6620-43c3-8319-eedd157d2251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:24.664986 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-5022d70a-6620-43c3-8319-eedd157d2251 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:24.666538 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5022d70a-6620-43c3-8319-eedd157d2251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:24.744337 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 141/278] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:25.070026 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d3a32-0ebd-46d9-a4a3-5a5db0cecdf0 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:25.070526 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d3a32-0ebd-46d9-a4a3-5a5db0cecdf0 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:25.070624 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d3a32-0ebd-46d9-a4a3-5a5db0cecdf0 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:25.072245 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 138/279] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:25.077244 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300ebd8-bcd8-4182-a4a0-16b06285d4ac None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:25.077531 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300ebd8-bcd8-4182-a4a0-16b06285d4ac None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:25.077802 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300ebd8-bcd8-4182-a4a0-16b06285d4ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:25.103343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-7300ebd8-bcd8-4182-a4a0-16b06285d4ac None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:25.104555 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7300ebd8-bcd8-4182-a4a0-16b06285d4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:25.161958 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:25.181914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 142/280] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:25 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 13 12:57:25.216126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41034b05-0442-4b37-87e1-4e00678fae48 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:25.216449 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41034b05-0442-4b37-87e1-4e00678fae48 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:25.216680 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41034b05-0442-4b37-87e1-4e00678fae48 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:25.218042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 139/281] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:25.224200 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f502a-ae05-4e4b-a3af-9ce89181a3bf None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:25.224489 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f502a-ae05-4e4b-a3af-9ce89181a3bf None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:25.224766 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f502a-ae05-4e4b-a3af-9ce89181a3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:25.252133 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-041f502a-ae05-4e4b-a3af-9ce89181a3bf None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:25.253363 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-041f502a-ae05-4e4b-a3af-9ce89181a3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:25.261543 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 143/282] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:25 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 13 12:57:25.291781 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41034b05-0442-4b37-87e1-4e00678fae48 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:25.418802 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41034b05-0442-4b37-87e1-4e00678fae48 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:25.461988 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5472f4-5f54-45d7-9d5f-19420654fcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:25.462642 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5472f4-5f54-45d7-9d5f-19420654fcac None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:25.462921 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5472f4-5f54-45d7-9d5f-19420654fcac None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:25.463220 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5472f4-5f54-45d7-9d5f-19420654fcac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:25.477196 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5472f4-5f54-45d7-9d5f-19420654fcac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:25.482640 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-3c5472f4-5f54-45d7-9d5f-19420654fcac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 13 12:57:25.482640 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:25.482640 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:25.482640 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:25.482640 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:25.482640 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:25.482640 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:25.482640 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:25.482640 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:25.482640 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:25.482640 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:25.482640 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:25.482640 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:25.482640 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:25.482640 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:25.482640 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:25.483524 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:25.483524 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:25.483524 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:25.483524 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:25.483524 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:25.483524 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:25.483524 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:25.483524 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:25.483524 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:25.483524 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:25.483524 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:25.483524 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:25.483524 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:25.483524 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:25.483524 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:25.483524 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:25.484375 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:25.484375 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:25.484375 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:25.484375 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:25.484375 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:25.484375 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:25.484375 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:25.484375 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:25.484375 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:25.484375 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:25.484375 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 13 12:57:25.484375 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:25.484375 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 140/283] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:57:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:25.491194 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f502a-ae05-4e4b-a3af-9ce89181a3bf None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:25.564324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5472f4-5f54-45d7-9d5f-19420654fcac None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:25.592199 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5472f4-5f54-45d7-9d5f-19420654fcac None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:25.605871 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1878198-c040-4c7a-b37d-2f41d58b41e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:25.606622 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1878198-c040-4c7a-b37d-2f41d58b41e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:25.606906 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1878198-c040-4c7a-b37d-2f41d58b41e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:25.607205 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1878198-c040-4c7a-b37d-2f41d58b41e3 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:25.631608 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1878198-c040-4c7a-b37d-2f41d58b41e3 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:25.640662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-c1878198-c040-4c7a-b37d-2f41d58b41e3 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 13 12:57:25.640662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:25.640662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:25.640662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:25.640662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:25.640662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:25.640662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:25.640662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:25.640662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:25.640662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:25.640662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:25.640662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:25.640662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:25.640662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:25.640662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:25.640662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:25.642684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:25.642684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:25.642684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:25.642684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:25.642684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:25.642684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:25.642684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:25.642684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:25.642684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:25.642684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:25.642684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:25.642684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:25.642684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:25.642684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:25.642684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:25.642684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:25.644601 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:25.644601 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:25.644601 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:25.644601 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:25.644601 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:25.644601 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:25.644601 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:25.644601 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:25.644601 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:25.644601 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:25.644601 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 13 12:57:25.644601 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:25.644601 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 141/284] 10.222.0.23 () {56 vars in 1135 bytes} [Thu Oct 13 12:57:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 79 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:25.652013 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1878198-c040-4c7a-b37d-2f41d58b41e3 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:25.668088 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f502a-ae05-4e4b-a3af-9ce89181a3bf None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:25.670554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1878198-c040-4c7a-b37d-2f41d58b41e3 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:25.681820 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3aadd3-8dca-435c-b04c-91709ae14cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:25.682488 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3aadd3-8dca-435c-b04c-91709ae14cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:25.682779 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3aadd3-8dca-435c-b04c-91709ae14cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:25.683077 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3aadd3-8dca-435c-b04c-91709ae14cd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:25.683809 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3aadd3-8dca-435c-b04c-91709ae14cd7 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:25.684107 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3aadd3-8dca-435c-b04c-91709ae14cd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:25.686336 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3aadd3-8dca-435c-b04c-91709ae14cd7 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:25.703279 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 142/285] 10.222.0.23 () {56 vars in 1132 bytes} [Thu Oct 13 12:57:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:25.705235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3aadd3-8dca-435c-b04c-91709ae14cd7 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:25.718467 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3aadd3-8dca-435c-b04c-91709ae14cd7 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:25.727610 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7b27b-90c3-44aa-a463-bc46f85f4a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:25.728231 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7b27b-90c3-44aa-a463-bc46f85f4a4c admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:25.728512 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7b27b-90c3-44aa-a463-bc46f85f4a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:25.728786 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7b27b-90c3-44aa-a463-bc46f85f4a4c admin admin] PATH_INFO: `/v3/groups/1969cf5ef4e344b1b4128ef7e86df9ad` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:25.732589 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc7b27b-90c3-44aa-a463-bc46f85f4a4c admin admin] RBAC: Authorizing `identity:get_group(group_id=1969cf5ef4e344b1b4128ef7e86df9ad)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:25.734624 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc7b27b-90c3-44aa-a463-bc46f85f4a4c admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:25.736585 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 143/286] 10.222.0.23 () {56 vars in 1201 bytes} [Thu Oct 13 12:57:25 2022] GET /identity/v3/groups/1969cf5ef4e344b1b4128ef7e86df9ad => generated 226 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:57:25.739934 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f4d57-0507-407d-9161-d2dceefbf6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:25.740782 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f4d57-0507-407d-9161-d2dceefbf6bf None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:25.741248 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f4d57-0507-407d-9161-d2dceefbf6bf None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:25.741704 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f4d57-0507-407d-9161-d2dceefbf6bf None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:25.742140 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7b27b-90c3-44aa-a463-bc46f85f4a4c admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:25.743050 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f4d57-0507-407d-9161-d2dceefbf6bf None admin] RBAC: Adding query filter params (name=reader) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:25.743571 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f4d57-0507-407d-9161-d2dceefbf6bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:25.745121 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f4d57-0507-407d-9161-d2dceefbf6bf None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:25.747571 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:25.755542 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7b27b-90c3-44aa-a463-bc46f85f4a4c admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:25.759398 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 144/287] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 13 12:57:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:25.764683 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39371048-7485-4055-a08c-672751af918a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:25.765327 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39371048-7485-4055-a08c-672751af918a admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:25.765641 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39371048-7485-4055-a08c-672751af918a admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:25.765922 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39371048-7485-4055-a08c-672751af918a admin admin] PATH_INFO: `/v3/projects/49ac31512d2943f2a28c22af2965292f` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:25.766274 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f4d57-0507-407d-9161-d2dceefbf6bf None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:25.767350 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39371048-7485-4055-a08c-672751af918a admin admin] RBAC: Authorizing `identity:get_project(project_id=49ac31512d2943f2a28c22af2965292f)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:25.769488 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39371048-7485-4055-a08c-672751af918a admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:25.775640 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 144/288] 10.222.0.23 () {56 vars in 1207 bytes} [Thu Oct 13 12:57:25 2022] GET /identity/v3/projects/49ac31512d2943f2a28c22af2965292f => generated 302 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:57:25.778074 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39371048-7485-4055-a08c-672751af918a admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:25.788373 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39371048-7485-4055-a08c-672751af918a admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:25.797525 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300a61b-183a-4510-997b-b6206e9ac958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:25.798235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300a61b-183a-4510-997b-b6206e9ac958 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:25.798515 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300a61b-183a-4510-997b-b6206e9ac958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:25.798845 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300a61b-183a-4510-997b-b6206e9ac958 admin admin] PATH_INFO: `/v3/projects/49ac31512d2943f2a28c22af2965292f/groups/1969cf5ef4e344b1b4128ef7e86df9ad/roles/5a10c87b17414f04846918d777506811` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:25.801957 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3300a61b-183a-4510-997b-b6206e9ac958 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49ac31512d2943f2a28c22af2965292f, group_id=1969cf5ef4e344b1b4128ef7e86df9ad, role_id=5a10c87b17414f04846918d777506811)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:25.804995 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3300a61b-183a-4510-997b-b6206e9ac958 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:25.826022 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 145/289] 10.222.0.23 () {58 vars in 1463 bytes} [Thu Oct 13 12:57:25 2022] PUT /identity/v3/projects/49ac31512d2943f2a28c22af2965292f/groups/1969cf5ef4e344b1b4128ef7e86df9ad/roles/5a10c87b17414f04846918d777506811 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 12:57:25.853580 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:25.923493 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f4d57-0507-407d-9161-d2dceefbf6bf None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:25.929758 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300a61b-183a-4510-997b-b6206e9ac958 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:25.989356 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca81558-107f-4263-a6c8-ac18f16e7e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:25.989997 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca81558-107f-4263-a6c8-ac18f16e7e74 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:25.990269 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca81558-107f-4263-a6c8-ac18f16e7e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:25.990538 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca81558-107f-4263-a6c8-ac18f16e7e74 None admin] PATH_INFO: `/v3/users/c0779e424b184b2994c8ee58c5a24706` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:25.991693 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca81558-107f-4263-a6c8-ac18f16e7e74 None admin] RBAC: Authorizing `identity:get_user(user_id=c0779e424b184b2994c8ee58c5a24706)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:25.993191 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca81558-107f-4263-a6c8-ac18f16e7e74 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:25.996414 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 145/290] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:57:25 2022] GET /identity/v3/users/c0779e424b184b2994c8ee58c5a24706 => generated 293 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:57:26.016855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca81558-107f-4263-a6c8-ac18f16e7e74 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:26.049944 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300a61b-183a-4510-997b-b6206e9ac958 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:26.060552 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ecf57-6285-4ab4-819c-ba1d14e4de99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:26.061257 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ecf57-6285-4ab4-819c-ba1d14e4de99 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:26.061819 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ecf57-6285-4ab4-819c-ba1d14e4de99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:26.062409 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ecf57-6285-4ab4-819c-ba1d14e4de99 admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:26.080583 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ecf57-6285-4ab4-819c-ba1d14e4de99 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:26.089632 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-f13ecf57-6285-4ab4-819c-ba1d14e4de99 admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:57:26.089632 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:26.089632 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:26.089632 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:26.089632 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:26.089632 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:26.089632 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:26.089632 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:26.089632 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:26.089632 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:26.089632 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:26.089632 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:26.089632 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:26.089632 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:26.089632 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:26.089632 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:26.090538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:26.090538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:26.090538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:26.090538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:26.090538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:26.090538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:26.090538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:26.090538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:26.090538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:26.090538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:26.090538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:26.090538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:26.090538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:26.090538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:26.090538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:26.090538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:26.091947 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:26.091947 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:26.091947 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:26.091947 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:26.091947 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:26.091947 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:26.091947 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:26.091947 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:26.091947 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:26.091947 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:26.091947 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:57:26.091947 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:26.095423 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 146/291] 10.222.0.23 () {56 vars in 1117 bytes} [Thu Oct 13 12:57:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:57:26.102157 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ecf57-6285-4ab4-819c-ba1d14e4de99 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:26.121176 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ecf57-6285-4ab4-819c-ba1d14e4de99 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:26.134304 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd44b4f-4c8d-4c9a-bc2f-bc1dee4ff10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:26.135654 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd44b4f-4c8d-4c9a-bc2f-bc1dee4ff10a admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:26.136222 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd44b4f-4c8d-4c9a-bc2f-bc1dee4ff10a admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:26.136457 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd44b4f-4c8d-4c9a-bc2f-bc1dee4ff10a admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:26.137143 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd44b4f-4c8d-4c9a-bc2f-bc1dee4ff10a admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:26.138837 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd44b4f-4c8d-4c9a-bc2f-bc1dee4ff10a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:26.140711 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd44b4f-4c8d-4c9a-bc2f-bc1dee4ff10a admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:26.156823 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca81558-107f-4263-a6c8-ac18f16e7e74 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:26.161894 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 147/292] 10.222.0.23 () {56 vars in 1120 bytes} [Thu Oct 13 12:57:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:26.173277 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd44b4f-4c8d-4c9a-bc2f-bc1dee4ff10a admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:26.191896 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd44b4f-4c8d-4c9a-bc2f-bc1dee4ff10a admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:26.197836 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:26.205673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072695bf-c6f9-455c-8970-7c8c99126d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:26.206364 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072695bf-c6f9-455c-8970-7c8c99126d20 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:26.206644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072695bf-c6f9-455c-8970-7c8c99126d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:26.208229 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072695bf-c6f9-455c-8970-7c8c99126d20 admin admin] PATH_INFO: `/v3/groups/a7ad9b87d0c144d1a54dc37a3a99defa` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:26.220294 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072695bf-c6f9-455c-8970-7c8c99126d20 admin admin] RBAC: Authorizing `identity:get_group(group_id=a7ad9b87d0c144d1a54dc37a3a99defa)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:26.225108 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072695bf-c6f9-455c-8970-7c8c99126d20 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:26.227045 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 148/293] 10.222.0.23 () {56 vars in 1201 bytes} [Thu Oct 13 12:57:26 2022] GET /identity/v3/groups/a7ad9b87d0c144d1a54dc37a3a99defa => generated 229 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:57:26.232001 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efbc273-a6bb-4e37-b4fc-ecce3748cfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:26.232722 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072695bf-c6f9-455c-8970-7c8c99126d20 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:26.233218 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efbc273-a6bb-4e37-b4fc-ecce3748cfac None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:26.233687 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efbc273-a6bb-4e37-b4fc-ecce3748cfac None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:26.234147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efbc273-a6bb-4e37-b4fc-ecce3748cfac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:26.235080 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efbc273-a6bb-4e37-b4fc-ecce3748cfac None admin] RBAC: Adding query filter params (role.id=4844dbf0f3e14acfaa7b2207a67377a7, user.id=c0779e424b184b2994c8ee58c5a24706, scope.system=all) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:26.235595 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efbc273-a6bb-4e37-b4fc-ecce3748cfac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:26.237198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efbc273-a6bb-4e37-b4fc-ecce3748cfac None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:26.243857 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072695bf-c6f9-455c-8970-7c8c99126d20 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:26.249279 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a0e51-7235-4255-b578-5dc702bde20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:26.249905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a0e51-7235-4255-b578-5dc702bde20c admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:26.250232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a0e51-7235-4255-b578-5dc702bde20c admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:26.250508 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a0e51-7235-4255-b578-5dc702bde20c admin admin] PATH_INFO: `/v3/projects/9cb9d18ca96e4beea199d190db853658` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:26.251707 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a0e51-7235-4255-b578-5dc702bde20c admin admin] RBAC: Authorizing `identity:get_project(project_id=9cb9d18ca96e4beea199d190db853658)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:26.253645 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69a0e51-7235-4255-b578-5dc702bde20c admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:26.254026 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 146/294] 10.222.0.23 () {56 vars in 1308 bytes} [Thu Oct 13 12:57:26 2022] GET /identity/v3/role_assignments?user.id=c0779e424b184b2994c8ee58c5a24706&scope.system=all&role.id=4844dbf0f3e14acfaa7b2207a67377a7 => generated 510 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:57:26.259128 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 149/295] 10.222.0.23 () {56 vars in 1207 bytes} [Thu Oct 13 12:57:26 2022] GET /identity/v3/projects/9cb9d18ca96e4beea199d190db853658 => generated 344 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:57:26.262109 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efbc273-a6bb-4e37-b4fc-ecce3748cfac None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:26.284293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efbc273-a6bb-4e37-b4fc-ecce3748cfac None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:26.292882 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f8357-89aa-41e1-921e-2d5a4d4a8b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:26.293661 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f8357-89aa-41e1-921e-2d5a4d4a8b75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:26.294042 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f8357-89aa-41e1-921e-2d5a4d4a8b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:26.294378 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f8357-89aa-41e1-921e-2d5a4d4a8b75 admin admin] PATH_INFO: `/v3/projects/9cb9d18ca96e4beea199d190db853658/groups/a7ad9b87d0c144d1a54dc37a3a99defa/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:26.296387 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f8357-89aa-41e1-921e-2d5a4d4a8b75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cb9d18ca96e4beea199d190db853658, group_id=a7ad9b87d0c144d1a54dc37a3a99defa, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:26.303264 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f8357-89aa-41e1-921e-2d5a4d4a8b75 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:26.322171 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 147/296] 10.222.0.23 () {58 vars in 1463 bytes} [Thu Oct 13 12:57:26 2022] PUT /identity/v3/projects/9cb9d18ca96e4beea199d190db853658/groups/a7ad9b87d0c144d1a54dc37a3a99defa/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 12:57:27.195701 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875dada-eaa4-4604-93e2-606eb58e538d None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:27.196674 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875dada-eaa4-4604-93e2-606eb58e538d None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:27.197155 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875dada-eaa4-4604-93e2-606eb58e538d None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:27.198918 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 150/297] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:27.207192 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92578acf-10fa-4e39-b7fb-733af8611541 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:27.208075 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92578acf-10fa-4e39-b7fb-733af8611541 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:27.208537 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92578acf-10fa-4e39-b7fb-733af8611541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:27.228903 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-92578acf-10fa-4e39-b7fb-733af8611541 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:27.230445 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-92578acf-10fa-4e39-b7fb-733af8611541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:27.306549 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 148/298] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 13 12:57:27.597017 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b1fb4-5b9b-4e38-a20e-57dcfb3cf692 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:27.597389 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b1fb4-5b9b-4e38-a20e-57dcfb3cf692 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:27.597564 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b1fb4-5b9b-4e38-a20e-57dcfb3cf692 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:27.598909 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 151/299] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:27.604582 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b3d1c-11fd-465d-9f7f-721fff93b10a None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:27.604872 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b3d1c-11fd-465d-9f7f-721fff93b10a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:27.605147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b3d1c-11fd-465d-9f7f-721fff93b10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:27.626670 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d35b3d1c-11fd-465d-9f7f-721fff93b10a None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:27.628546 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d35b3d1c-11fd-465d-9f7f-721fff93b10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:27.629310 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b9771-45b2-456d-9ba6-4acc374becc5 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:27.629600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b9771-45b2-456d-9ba6-4acc374becc5 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:27.629875 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b9771-45b2-456d-9ba6-4acc374becc5 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:27.631500 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 152/300] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:27.644522 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff7ab9-85d5-4f5f-8db8-986ac2dbec83 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:27.644816 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff7ab9-85d5-4f5f-8db8-986ac2dbec83 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:27.645083 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff7ab9-85d5-4f5f-8db8-986ac2dbec83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:27.671917 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-bdff7ab9-85d5-4f5f-8db8-986ac2dbec83 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:27.673122 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-bdff7ab9-85d5-4f5f-8db8-986ac2dbec83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:27.708873 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 149/301] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 13 12:57:27.710353 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b3d1c-11fd-465d-9f7f-721fff93b10a None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:27.742508 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:27.758526 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 153/302] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 13 12:57:27.827549 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b3d1c-11fd-465d-9f7f-721fff93b10a None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:27.863926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232db623-cd87-4d31-813b-278efa0f2a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:27.864887 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232db623-cd87-4d31-813b-278efa0f2a5f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:27.865354 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232db623-cd87-4d31-813b-278efa0f2a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:27.865810 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232db623-cd87-4d31-813b-278efa0f2a5f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:27.877299 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232db623-cd87-4d31-813b-278efa0f2a5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:27.882270 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-232db623-cd87-4d31-813b-278efa0f2a5f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 13 12:57:27.882270 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:27.882270 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:27.882270 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:27.882270 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:27.882270 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:27.882270 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:27.882270 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:27.882270 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:27.882270 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:27.882270 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:27.882270 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:27.882270 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:27.882270 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:27.882270 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:27.882270 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:27.883147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:27.883147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:27.883147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:27.883147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:27.883147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:27.883147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:27.883147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:27.883147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:27.883147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:27.883147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:27.883147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:27.883147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:27.883147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:27.883147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:27.883147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:27.883147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:27.883929 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:27.883929 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:27.883929 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:27.883929 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:27.883929 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:27.883929 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:27.883929 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:27.883929 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:27.883929 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:27.883929 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:27.883929 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 13 12:57:27.883929 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:27.886115 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 150/303] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Oct 13 12:57:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:27.892587 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff7ab9-85d5-4f5f-8db8-986ac2dbec83 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:27.962198 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:28.041781 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff7ab9-85d5-4f5f-8db8-986ac2dbec83 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:28.083980 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e3a03-cb16-4a80-be2b-4a62b0ab3c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:28.084593 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e3a03-cb16-4a80-be2b-4a62b0ab3c20 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:28.084873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e3a03-cb16-4a80-be2b-4a62b0ab3c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:28.085143 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e3a03-cb16-4a80-be2b-4a62b0ab3c20 None admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:28.085809 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e3a03-cb16-4a80-be2b-4a62b0ab3c20 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:28.086111 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e3a03-cb16-4a80-be2b-4a62b0ab3c20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:28.087507 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e3a03-cb16-4a80-be2b-4a62b0ab3c20 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:28.098447 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 154/304] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:57:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:28.099927 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232db623-cd87-4d31-813b-278efa0f2a5f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:28.104004 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e3a03-cb16-4a80-be2b-4a62b0ab3c20 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:28.210922 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:28.236714 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232db623-cd87-4d31-813b-278efa0f2a5f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:28.244515 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:28.256001 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e3a03-cb16-4a80-be2b-4a62b0ab3c20 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:28.288015 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503ccf2c-3e23-44e4-baa5-df2b9b6c4573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:28.288650 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ccf2c-3e23-44e4-baa5-df2b9b6c4573 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:28.289184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ccf2c-3e23-44e4-baa5-df2b9b6c4573 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:28.289519 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503ccf2c-3e23-44e4-baa5-df2b9b6c4573 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:28.295659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04205f1a-ce29-425b-b1fc-3d4c67f82330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:28.296275 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04205f1a-ce29-425b-b1fc-3d4c67f82330 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:28.296556 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04205f1a-ce29-425b-b1fc-3d4c67f82330 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:28.296827 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04205f1a-ce29-425b-b1fc-3d4c67f82330 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:28.304833 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503ccf2c-3e23-44e4-baa5-df2b9b6c4573 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:28.311166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-503ccf2c-3e23-44e4-baa5-df2b9b6c4573 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 13 12:57:28.311166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:28.311166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:28.311166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:28.311166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:28.311166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:28.311166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:28.311166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:28.311166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:28.311166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:28.311166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:28.311166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:28.311166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:28.311166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:28.311166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:28.311166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:28.312509 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:28.312509 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:28.312509 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:28.312509 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:28.312509 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:28.312509 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:28.312509 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:28.312509 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:28.312509 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:28.312509 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:28.312509 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:28.312509 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:28.312509 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:28.312509 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:28.312509 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:28.312509 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:28.313472 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:28.313472 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:28.313472 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:28.313472 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:28.313472 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:28.313472 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:28.313472 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:28.313472 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:28.313472 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:28.313472 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:28.313472 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 13 12:57:28.313472 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:28.321022 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 151/305] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:57:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:28.321022 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503ccf2c-3e23-44e4-baa5-df2b9b6c4573 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:28.393311 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04205f1a-ce29-425b-b1fc-3d4c67f82330 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:28.394883 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04205f1a-ce29-425b-b1fc-3d4c67f82330 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:28.438264 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503ccf2c-3e23-44e4-baa5-df2b9b6c4573 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:28.478993 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa6fe0-8eaf-4d47-80a5-6d9a9de00ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:28.482439 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa6fe0-8eaf-4d47-80a5-6d9a9de00ae7 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:28.482729 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa6fe0-8eaf-4d47-80a5-6d9a9de00ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:28.483003 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa6fe0-8eaf-4d47-80a5-6d9a9de00ae7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:28.496315 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa6fe0-8eaf-4d47-80a5-6d9a9de00ae7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:28.499487 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:28.504398 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-6eaa6fe0-8eaf-4d47-80a5-6d9a9de00ae7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 13 12:57:28.504398 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:28.504398 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:28.504398 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:28.504398 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:28.504398 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:28.504398 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:28.504398 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:28.504398 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:28.504398 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:28.504398 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:28.504398 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:28.504398 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:28.504398 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:28.504398 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:28.504398 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:28.505760 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:28.505760 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:28.505760 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:28.505760 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:28.505760 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:28.505760 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:28.505760 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:28.505760 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:28.505760 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:28.505760 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:28.505760 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:28.505760 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:28.505760 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:28.505760 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:28.505760 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:28.505760 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:28.506763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:28.506763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:28.506763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:28.506763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:28.506763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:28.506763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:28.506763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:28.506763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:28.506763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:28.506763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:28.506763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 13 12:57:28.506763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:28.507508 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 155/306] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 13 12:57:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 13 12:57:28.507930 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04205f1a-ce29-425b-b1fc-3d4c67f82330 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:28.510442 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 152/307] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:57:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:28.513147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa6fe0-8eaf-4d47-80a5-6d9a9de00ae7 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:28.519979 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:28.628117 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04205f1a-ce29-425b-b1fc-3d4c67f82330 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:28.642006 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa6fe0-8eaf-4d47-80a5-6d9a9de00ae7 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:28.665692 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1604dceb-9fd3-48c0-98e7-b52ac84eae45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:28.666317 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604dceb-9fd3-48c0-98e7-b52ac84eae45 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:28.666600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604dceb-9fd3-48c0-98e7-b52ac84eae45 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:28.666871 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604dceb-9fd3-48c0-98e7-b52ac84eae45 None admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:28.667568 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1604dceb-9fd3-48c0-98e7-b52ac84eae45 None admin] RBAC: Adding query filter params (name=member) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:28.668140 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1604dceb-9fd3-48c0-98e7-b52ac84eae45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:28.669871 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1604dceb-9fd3-48c0-98e7-b52ac84eae45 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:28.679025 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 156/308] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 13 12:57:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:28.679913 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1604dceb-9fd3-48c0-98e7-b52ac84eae45 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:28.684237 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f066f7-c5cd-49c7-94c5-c009808b452e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:28.685131 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f066f7-c5cd-49c7-94c5-c009808b452e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:28.685581 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f066f7-c5cd-49c7-94c5-c009808b452e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:28.686025 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f066f7-c5cd-49c7-94c5-c009808b452e None admin] PATH_INFO: `/v3/projects/b14cb10007d440b4828ccc9869dc8aa2` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:28.687669 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f066f7-c5cd-49c7-94c5-c009808b452e None admin] RBAC: Authorizing `identity:get_project(project_id=b14cb10007d440b4828ccc9869dc8aa2)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:28.690298 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f066f7-c5cd-49c7-94c5-c009808b452e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:28.692803 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 153/309] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 13 12:57:28 2022] GET /identity/v3/projects/b14cb10007d440b4828ccc9869dc8aa2 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:28.694480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f066f7-c5cd-49c7-94c5-c009808b452e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:28.751966 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:28.782636 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:28.807381 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1604dceb-9fd3-48c0-98e7-b52ac84eae45 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:28.826830 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f066f7-c5cd-49c7-94c5-c009808b452e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:28.859483 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0391980-b02e-4722-a37c-f663600a3408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:28.860927 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0391980-b02e-4722-a37c-f663600a3408 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:28.861396 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0391980-b02e-4722-a37c-f663600a3408 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:28.861845 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0391980-b02e-4722-a37c-f663600a3408 None admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:28.863006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0391980-b02e-4722-a37c-f663600a3408 None admin] RBAC: Adding query filter params (name=member) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:28.863596 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0391980-b02e-4722-a37c-f663600a3408 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:28.866405 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0391980-b02e-4722-a37c-f663600a3408 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:28.876134 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3a381-f5f2-4b26-bf46-0a53eb60f166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:28.876863 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3a381-f5f2-4b26-bf46-0a53eb60f166 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:28.877213 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3a381-f5f2-4b26-bf46-0a53eb60f166 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:28.877556 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3a381-f5f2-4b26-bf46-0a53eb60f166 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:28.878634 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 157/310] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 13 12:57:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:28.879856 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0391980-b02e-4722-a37c-f663600a3408 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:28.975586 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e3a381-f5f2-4b26-bf46-0a53eb60f166 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:28.977418 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e3a381-f5f2-4b26-bf46-0a53eb60f166 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:29.020565 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0391980-b02e-4722-a37c-f663600a3408 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:29.042229 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:29.060382 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b5b01-7b48-41ad-8b38-f43f88a0542f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:29.061104 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b5b01-7b48-41ad-8b38-f43f88a0542f None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:29.061444 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b5b01-7b48-41ad-8b38-f43f88a0542f None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:29.061773 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b5b01-7b48-41ad-8b38-f43f88a0542f None admin] PATH_INFO: `/v3/groups/1969cf5ef4e344b1b4128ef7e86df9ad` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:29.063502 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b5b01-7b48-41ad-8b38-f43f88a0542f None admin] RBAC: Authorizing `identity:get_group(group_id=1969cf5ef4e344b1b4128ef7e86df9ad)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:29.065573 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b5b01-7b48-41ad-8b38-f43f88a0542f None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:29.067241 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 158/311] 10.222.0.23 () {56 vars in 1180 bytes} [Thu Oct 13 12:57:28 2022] GET /identity/v3/groups/1969cf5ef4e344b1b4128ef7e86df9ad => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:29.068075 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b5b01-7b48-41ad-8b38-f43f88a0542f None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:29.068780 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:29.118956 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 154/312] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 13 12:57:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 13 12:57:29.120567 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a937488-4c02-46db-8d88-3d207cbecc72 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:29.120856 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a937488-4c02-46db-8d88-3d207cbecc72 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:29.121118 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a937488-4c02-46db-8d88-3d207cbecc72 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:29.123035 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 155/313] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:29.123646 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a937488-4c02-46db-8d88-3d207cbecc72 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:29.206834 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b5b01-7b48-41ad-8b38-f43f88a0542f None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:29.243160 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a937488-4c02-46db-8d88-3d207cbecc72 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:29.251162 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdbbc4d-8dc0-4abe-9f2e-c73922a75648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:29.252798 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbbc4d-8dc0-4abe-9f2e-c73922a75648 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:29.253286 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbbc4d-8dc0-4abe-9f2e-c73922a75648 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:29.254122 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbbc4d-8dc0-4abe-9f2e-c73922a75648 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:29.289972 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52076a55-f492-4958-8e15-a0b0074694b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:29.290897 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52076a55-f492-4958-8e15-a0b0074694b5 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:29.291417 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52076a55-f492-4958-8e15-a0b0074694b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:29.291692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52076a55-f492-4958-8e15-a0b0074694b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:29.292457 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52076a55-f492-4958-8e15-a0b0074694b5 None admin] RBAC: Adding query filter params (group.id=1969cf5ef4e344b1b4128ef7e86df9ad, role.id=5a10c87b17414f04846918d777506811, scope.project.id=b14cb10007d440b4828ccc9869dc8aa2) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:29.293350 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52076a55-f492-4958-8e15-a0b0074694b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:29.294301 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52076a55-f492-4958-8e15-a0b0074694b5 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:29.302202 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 156/314] 10.222.0.23 () {56 vars in 1376 bytes} [Thu Oct 13 12:57:29 2022] GET /identity/v3/role_assignments?group.id=1969cf5ef4e344b1b4128ef7e86df9ad&scope.project.id=b14cb10007d440b4828ccc9869dc8aa2&role.id=5a10c87b17414f04846918d777506811 => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:29.302202 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52076a55-f492-4958-8e15-a0b0074694b5 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:29.340776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:29.346102 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:29.354083 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdbbc4d-8dc0-4abe-9f2e-c73922a75648 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:29.356063 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdbbc4d-8dc0-4abe-9f2e-c73922a75648 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:29.430520 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52076a55-f492-4958-8e15-a0b0074694b5 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:29.479601 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf5af73-b5ca-4825-84d5-1c6f9b9b70df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:29.480487 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5af73-b5ca-4825-84d5-1c6f9b9b70df None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:29.480593 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5af73-b5ca-4825-84d5-1c6f9b9b70df None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:29.480802 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5af73-b5ca-4825-84d5-1c6f9b9b70df None admin] PATH_INFO: `/v3/projects/b14cb10007d440b4828ccc9869dc8aa2` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:29.482593 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf5af73-b5ca-4825-84d5-1c6f9b9b70df None admin] RBAC: Authorizing `identity:get_project(project_id=b14cb10007d440b4828ccc9869dc8aa2)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:29.484087 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf5af73-b5ca-4825-84d5-1c6f9b9b70df None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:29.486615 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 157/315] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 13 12:57:29 2022] GET /identity/v3/projects/b14cb10007d440b4828ccc9869dc8aa2 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:29.487703 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4356f-6f23-45d3-a021-530ca1f9e190 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:29.487990 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4356f-6f23-45d3-a021-530ca1f9e190 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:29.488338 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4356f-6f23-45d3-a021-530ca1f9e190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:29.494330 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 159/316] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 13 12:57:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 13 12:57:29.495841 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62e6b6-f2ae-4eaa-b094-3f374cb17ddc None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:29.496198 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62e6b6-f2ae-4eaa-b094-3f374cb17ddc None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:29.496538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62e6b6-f2ae-4eaa-b094-3f374cb17ddc None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:29.498776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 160/317] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:29.499665 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62e6b6-f2ae-4eaa-b094-3f374cb17ddc None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:29.508510 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-aab4356f-6f23-45d3-a021-530ca1f9e190 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:29.509444 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-aab4356f-6f23-45d3-a021-530ca1f9e190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:29.590712 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 158/318] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:29.595744 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4356f-6f23-45d3-a021-530ca1f9e190 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:29.608536 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:29.624305 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62e6b6-f2ae-4eaa-b094-3f374cb17ddc None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:29.641304 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:29.666075 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6358ca3-a314-4d96-9862-8850b0a28ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:29.666722 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6358ca3-a314-4d96-9862-8850b0a28ddd None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:29.667001 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6358ca3-a314-4d96-9862-8850b0a28ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:29.667319 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6358ca3-a314-4d96-9862-8850b0a28ddd None admin] PATH_INFO: `/v3/groups/1969cf5ef4e344b1b4128ef7e86df9ad` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:29.668431 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6358ca3-a314-4d96-9862-8850b0a28ddd None admin] RBAC: Authorizing `identity:get_group(group_id=1969cf5ef4e344b1b4128ef7e86df9ad)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:29.670419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6358ca3-a314-4d96-9862-8850b0a28ddd None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:29.672604 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 161/319] 10.222.0.23 () {56 vars in 1180 bytes} [Thu Oct 13 12:57:29 2022] GET /identity/v3/groups/1969cf5ef4e344b1b4128ef7e86df9ad => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:29.673650 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba4b01-cda7-4abd-a98a-8760a8d19075 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:29.674010 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba4b01-cda7-4abd-a98a-8760a8d19075 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:29.674263 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba4b01-cda7-4abd-a98a-8760a8d19075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:29.692931 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-66ba4b01-cda7-4abd-a98a-8760a8d19075 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:29.694122 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-66ba4b01-cda7-4abd-a98a-8760a8d19075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:29.728472 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4356f-6f23-45d3-a021-530ca1f9e190 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:29.770896 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1dea0-23f1-4c0e-ab44-957c258632a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:29.771692 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 162/320] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:29.772118 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1dea0-23f1-4c0e-ab44-957c258632a5 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:29.772579 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba4b01-cda7-4abd-a98a-8760a8d19075 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:29.772809 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1dea0-23f1-4c0e-ab44-957c258632a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:29.773266 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1dea0-23f1-4c0e-ab44-957c258632a5 None admin] PATH_INFO: `/v3/projects/49ac31512d2943f2a28c22af2965292f` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:29.775742 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b1dea0-23f1-4c0e-ab44-957c258632a5 None admin] RBAC: Authorizing `identity:get_project(project_id=49ac31512d2943f2a28c22af2965292f)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:29.777381 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b1dea0-23f1-4c0e-ab44-957c258632a5 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:29.779257 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 159/321] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 13 12:57:29 2022] GET /identity/v3/projects/49ac31512d2943f2a28c22af2965292f => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:29.786168 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1dea0-23f1-4c0e-ab44-957c258632a5 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:29.882767 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:29.898330 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba4b01-cda7-4abd-a98a-8760a8d19075 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:29.903592 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:29.914135 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1dea0-23f1-4c0e-ab44-957c258632a5 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:29.942936 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a60478-cfed-4a41-9120-37d586ff2f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:29.943591 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a60478-cfed-4a41-9120-37d586ff2f6d None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:29.943871 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a60478-cfed-4a41-9120-37d586ff2f6d None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:29.944145 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a60478-cfed-4a41-9120-37d586ff2f6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:29.944983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a60478-cfed-4a41-9120-37d586ff2f6d None admin] RBAC: Adding query filter params (group.id=1969cf5ef4e344b1b4128ef7e86df9ad, role.id=10ba22bff4d24fd69e4c6def38370617, scope.project.id=b14cb10007d440b4828ccc9869dc8aa2) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:29.945296 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a60478-cfed-4a41-9120-37d586ff2f6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:29.946841 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a60478-cfed-4a41-9120-37d586ff2f6d None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:29.954513 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 163/322] 10.222.0.23 () {56 vars in 1376 bytes} [Thu Oct 13 12:57:29 2022] GET /identity/v3/role_assignments?group.id=1969cf5ef4e344b1b4128ef7e86df9ad&scope.project.id=b14cb10007d440b4828ccc9869dc8aa2&role.id=10ba22bff4d24fd69e4c6def38370617 => generated 611 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:29.955152 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a60478-cfed-4a41-9120-37d586ff2f6d None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:29.959453 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ed008a-479b-4f7f-92af-b9d047c535e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:29.960382 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed008a-479b-4f7f-92af-b9d047c535e5 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:29.960836 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed008a-479b-4f7f-92af-b9d047c535e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:29.961279 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed008a-479b-4f7f-92af-b9d047c535e5 None admin] PATH_INFO: `/v3/groups/1969cf5ef4e344b1b4128ef7e86df9ad` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:29.962683 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ed008a-479b-4f7f-92af-b9d047c535e5 None admin] RBAC: Authorizing `identity:get_group(group_id=1969cf5ef4e344b1b4128ef7e86df9ad)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:29.964550 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ed008a-479b-4f7f-92af-b9d047c535e5 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:29.967128 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 160/323] 10.222.0.23 () {56 vars in 1180 bytes} [Thu Oct 13 12:57:29 2022] GET /identity/v3/groups/1969cf5ef4e344b1b4128ef7e86df9ad => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:57:29.974042 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ed008a-479b-4f7f-92af-b9d047c535e5 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:30.082517 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a60478-cfed-4a41-9120-37d586ff2f6d None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:30.119157 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ed008a-479b-4f7f-92af-b9d047c535e5 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:30.131270 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5dd07d-2a4d-4a5c-977f-1db200998cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:30.132219 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5dd07d-2a4d-4a5c-977f-1db200998cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:30.132643 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5dd07d-2a4d-4a5c-977f-1db200998cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:30.133122 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5dd07d-2a4d-4a5c-977f-1db200998cd1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:30.146729 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5dd07d-2a4d-4a5c-977f-1db200998cd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:30.155569 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-2f5dd07d-2a4d-4a5c-977f-1db200998cd1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 13 12:57:30.155569 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:30.155569 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:30.155569 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:30.155569 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:30.155569 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:30.155569 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:30.155569 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:30.155569 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:30.155569 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:30.155569 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:30.155569 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:30.155569 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:30.155569 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:30.155569 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:30.155569 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:30.156502 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:30.156502 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:30.156502 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:30.156502 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:30.156502 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:30.156502 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:30.156502 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:30.156502 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:30.156502 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:30.156502 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:30.156502 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:30.156502 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:30.156502 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:30.156502 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:30.156502 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:30.156502 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:30.157962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:30.157962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:30.157962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:30.157962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:30.157962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:30.157962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:30.157962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:30.157962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:30.157962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:30.157962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:30.157962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 13 12:57:30.157962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:30.157962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 164/324] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Oct 13 12:57:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:30.157962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5dd07d-2a4d-4a5c-977f-1db200998cd1 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:30.164770 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d96f2ab-d441-4e62-a9bd-47a964c46ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:30.165386 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96f2ab-d441-4e62-a9bd-47a964c46ebc None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:30.165662 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96f2ab-d441-4e62-a9bd-47a964c46ebc None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:30.165930 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96f2ab-d441-4e62-a9bd-47a964c46ebc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:30.166664 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d96f2ab-d441-4e62-a9bd-47a964c46ebc None admin] RBAC: Adding query filter params (group.id=1969cf5ef4e344b1b4128ef7e86df9ad, role.id=10ba22bff4d24fd69e4c6def38370617, scope.project.id=49ac31512d2943f2a28c22af2965292f) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:30.166970 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d96f2ab-d441-4e62-a9bd-47a964c46ebc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:30.169196 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d96f2ab-d441-4e62-a9bd-47a964c46ebc None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:30.172788 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:30.176474 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 161/325] 10.222.0.23 () {56 vars in 1376 bytes} [Thu Oct 13 12:57:29 2022] GET /identity/v3/role_assignments?group.id=1969cf5ef4e344b1b4128ef7e86df9ad&scope.project.id=49ac31512d2943f2a28c22af2965292f&role.id=10ba22bff4d24fd69e4c6def38370617 => generated 611 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:57:30.177311 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d96f2ab-d441-4e62-a9bd-47a964c46ebc None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:30.222238 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:30.295668 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5dd07d-2a4d-4a5c-977f-1db200998cd1 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:30.319814 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d96f2ab-d441-4e62-a9bd-47a964c46ebc None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:30.342217 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd99abe-351c-491d-85c1-a244f5e27d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:30.342871 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd99abe-351c-491d-85c1-a244f5e27d03 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:30.343219 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd99abe-351c-491d-85c1-a244f5e27d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:30.343494 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd99abe-351c-491d-85c1-a244f5e27d03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:30.357894 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd99abe-351c-491d-85c1-a244f5e27d03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:30.362529 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5f0a0-2bc4-4b51-9aac-264807e2e7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:30.363199 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5f0a0-2bc4-4b51-9aac-264807e2e7bc None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:30.363385 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-6dd99abe-351c-491d-85c1-a244f5e27d03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:57:30.363385 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:30.363385 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:30.363385 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:30.363385 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:30.363385 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:30.363385 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:30.363385 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:30.363385 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:30.363385 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:30.363385 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:30.363385 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:30.363385 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:30.363385 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:30.363385 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:30.363385 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:30.364232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:30.364232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:30.364232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:30.364232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:30.364232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:30.364232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:30.364232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:30.364232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:30.364232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:30.364232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:30.364232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:30.364232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:30.364232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:30.364232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:30.364232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:30.364232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:30.365030 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:30.365030 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:30.365030 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:30.365030 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:30.365030 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:30.365030 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:30.365030 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:30.365030 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:30.365030 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:30.365030 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:30.365030 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:57:30.365030 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:30.365609 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5f0a0-2bc4-4b51-9aac-264807e2e7bc None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:30.365609 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5f0a0-2bc4-4b51-9aac-264807e2e7bc None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:30.365609 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e5f0a0-2bc4-4b51-9aac-264807e2e7bc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:30.365609 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e5f0a0-2bc4-4b51-9aac-264807e2e7bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:30.365830 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 165/326] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 13 12:57:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:30.367774 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e5f0a0-2bc4-4b51-9aac-264807e2e7bc None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:30.375181 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd99abe-351c-491d-85c1-a244f5e27d03 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:30.377414 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 162/327] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:57:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:30.383805 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5f0a0-2bc4-4b51-9aac-264807e2e7bc None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:30.466808 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:30.502014 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd99abe-351c-491d-85c1-a244f5e27d03 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:30.503805 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:30.512097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5f0a0-2bc4-4b51-9aac-264807e2e7bc None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:30.544753 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65e525-f0be-401c-a56a-9c7d50a33e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:30.545953 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65e525-f0be-401c-a56a-9c7d50a33e20 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:30.546390 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65e525-f0be-401c-a56a-9c7d50a33e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:30.546936 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65e525-f0be-401c-a56a-9c7d50a33e20 None admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:30.547904 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65e525-f0be-401c-a56a-9c7d50a33e20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:30.548213 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65e525-f0be-401c-a56a-9c7d50a33e20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:30.549931 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65e525-f0be-401c-a56a-9c7d50a33e20 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:30.552962 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f08e749-af17-4836-8f1e-2acdef7f99c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:30.553573 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08e749-af17-4836-8f1e-2acdef7f99c3 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:30.553849 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08e749-af17-4836-8f1e-2acdef7f99c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:30.554119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08e749-af17-4836-8f1e-2acdef7f99c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:30.558844 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 166/328] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:57:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:57:30.565195 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65e525-f0be-401c-a56a-9c7d50a33e20 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:30.636131 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f08e749-af17-4836-8f1e-2acdef7f99c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:30.637539 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f08e749-af17-4836-8f1e-2acdef7f99c3 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:30.682601 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65e525-f0be-401c-a56a-9c7d50a33e20 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:30.716301 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:30.723039 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16123ee-da97-455f-b0ab-82ce8cef6aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:30.723916 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16123ee-da97-455f-b0ab-82ce8cef6aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:30.724295 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16123ee-da97-455f-b0ab-82ce8cef6aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:30.724572 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16123ee-da97-455f-b0ab-82ce8cef6aa9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:30.752651 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:30.761203 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 163/329] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 13 12:57:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 13 12:57:30.766855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f08e749-af17-4836-8f1e-2acdef7f99c3 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:30.806737 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16123ee-da97-455f-b0ab-82ce8cef6aa9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:30.808321 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16123ee-da97-455f-b0ab-82ce8cef6aa9 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:30.882499 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f08e749-af17-4836-8f1e-2acdef7f99c3 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:30.916871 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc6633-9f8a-4c09-9be7-716d92805c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:30.917504 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc6633-9f8a-4c09-9be7-716d92805c98 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:30.917786 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc6633-9f8a-4c09-9be7-716d92805c98 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:30.918058 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc6633-9f8a-4c09-9be7-716d92805c98 None admin] PATH_INFO: `/v3/projects/49ac31512d2943f2a28c22af2965292f` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:30.919984 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc6633-9f8a-4c09-9be7-716d92805c98 None admin] RBAC: Authorizing `identity:get_project(project_id=49ac31512d2943f2a28c22af2965292f)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:30.921479 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc6633-9f8a-4c09-9be7-716d92805c98 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:30.927410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 167/330] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 13 12:57:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 13 12:57:30.927566 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 164/331] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 13 12:57:30 2022] GET /identity/v3/projects/49ac31512d2943f2a28c22af2965292f => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:57:30.928416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc6633-9f8a-4c09-9be7-716d92805c98 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:30.928534 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16123ee-da97-455f-b0ab-82ce8cef6aa9 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:30.958642 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:30.988051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:31.025177 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16123ee-da97-455f-b0ab-82ce8cef6aa9 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:31.025678 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc6633-9f8a-4c09-9be7-716d92805c98 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:31.057442 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a87a4-654b-4b33-9c24-4bf321779b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:31.057955 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c107b4-ab0f-4ea6-bf95-934d4dc13760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:31.058072 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a87a4-654b-4b33-9c24-4bf321779b17 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:31.058324 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a87a4-654b-4b33-9c24-4bf321779b17 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:31.058564 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c107b4-ab0f-4ea6-bf95-934d4dc13760 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:31.058659 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a87a4-654b-4b33-9c24-4bf321779b17 None admin] PATH_INFO: `/v3/groups/1969cf5ef4e344b1b4128ef7e86df9ad` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:31.058838 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c107b4-ab0f-4ea6-bf95-934d4dc13760 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:31.059151 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c107b4-ab0f-4ea6-bf95-934d4dc13760 None admin] PATH_INFO: `/v3/projects/9cb9d18ca96e4beea199d190db853658` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:31.060233 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26a87a4-654b-4b33-9c24-4bf321779b17 None admin] RBAC: Authorizing `identity:get_group(group_id=1969cf5ef4e344b1b4128ef7e86df9ad)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:31.060770 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c107b4-ab0f-4ea6-bf95-934d4dc13760 None admin] RBAC: Authorizing `identity:get_project(project_id=9cb9d18ca96e4beea199d190db853658)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:31.061654 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26a87a4-654b-4b33-9c24-4bf321779b17 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:31.062436 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c107b4-ab0f-4ea6-bf95-934d4dc13760 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:31.063501 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 165/332] 10.222.0.23 () {56 vars in 1180 bytes} [Thu Oct 13 12:57:30 2022] GET /identity/v3/groups/1969cf5ef4e344b1b4128ef7e86df9ad => generated 226 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:31.064034 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 168/333] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 13 12:57:30 2022] GET /identity/v3/projects/9cb9d18ca96e4beea199d190db853658 => generated 344 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:31.069241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a87a4-654b-4b33-9c24-4bf321779b17 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:31.069444 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c107b4-ab0f-4ea6-bf95-934d4dc13760 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:31.160847 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:31.164051 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c107b4-ab0f-4ea6-bf95-934d4dc13760 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:31.166083 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a87a4-654b-4b33-9c24-4bf321779b17 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:31.195141 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:31.199530 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b176d3a7-f03a-4c2b-9dad-fefdd8d13188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:31.200157 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176d3a7-f03a-4c2b-9dad-fefdd8d13188 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:31.200438 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176d3a7-f03a-4c2b-9dad-fefdd8d13188 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:31.200711 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176d3a7-f03a-4c2b-9dad-fefdd8d13188 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:31.201131 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a433f0-4178-4991-b761-ff8f8fecc774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:31.201434 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b176d3a7-f03a-4c2b-9dad-fefdd8d13188 None admin] RBAC: Adding query filter params (group.id=1969cf5ef4e344b1b4128ef7e86df9ad, role.id=5a10c87b17414f04846918d777506811, scope.project.id=49ac31512d2943f2a28c22af2965292f) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:31.201744 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b176d3a7-f03a-4c2b-9dad-fefdd8d13188 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:31.201835 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a433f0-4178-4991-b761-ff8f8fecc774 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:31.202018 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a433f0-4178-4991-b761-ff8f8fecc774 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:31.202286 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a433f0-4178-4991-b761-ff8f8fecc774 None admin] PATH_INFO: `/v3/groups/a7ad9b87d0c144d1a54dc37a3a99defa` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:31.203189 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b176d3a7-f03a-4c2b-9dad-fefdd8d13188 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:31.203408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a433f0-4178-4991-b761-ff8f8fecc774 None admin] RBAC: Authorizing `identity:get_group(group_id=a7ad9b87d0c144d1a54dc37a3a99defa)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:31.204811 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a433f0-4178-4991-b761-ff8f8fecc774 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:31.206370 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 166/334] 10.222.0.23 () {56 vars in 1180 bytes} [Thu Oct 13 12:57:31 2022] GET /identity/v3/groups/a7ad9b87d0c144d1a54dc37a3a99defa => generated 229 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:31.208871 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 169/335] 10.222.0.23 () {56 vars in 1376 bytes} [Thu Oct 13 12:57:31 2022] GET /identity/v3/role_assignments?group.id=1969cf5ef4e344b1b4128ef7e86df9ad&scope.project.id=49ac31512d2943f2a28c22af2965292f&role.id=5a10c87b17414f04846918d777506811 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:57:31.212626 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a433f0-4178-4991-b761-ff8f8fecc774 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:31.316374 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a433f0-4178-4991-b761-ff8f8fecc774 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:31.350394 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac0b36-649d-4525-aade-062103a6049b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:31.351091 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac0b36-649d-4525-aade-062103a6049b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:31.351304 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac0b36-649d-4525-aade-062103a6049b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:31.351574 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac0b36-649d-4525-aade-062103a6049b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:31.352312 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac0b36-649d-4525-aade-062103a6049b None admin] RBAC: Adding query filter params (group.id=a7ad9b87d0c144d1a54dc37a3a99defa, role.id=70af94fc6a8149f2a38f74b843e1a9f0, scope.project.id=9cb9d18ca96e4beea199d190db853658) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:31.352624 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac0b36-649d-4525-aade-062103a6049b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:31.354054 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac0b36-649d-4525-aade-062103a6049b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:31.360898 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 167/336] 10.222.0.23 () {56 vars in 1376 bytes} [Thu Oct 13 12:57:31 2022] GET /identity/v3/role_assignments?group.id=a7ad9b87d0c144d1a54dc37a3a99defa&scope.project.id=9cb9d18ca96e4beea199d190db853658&role.id=70af94fc6a8149f2a38f74b843e1a9f0 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:57:33.881329 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9dee21-999c-4780-99a3-83f56a7f61bc None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:33.881849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9dee21-999c-4780-99a3-83f56a7f61bc None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:33.882191 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9dee21-999c-4780-99a3-83f56a7f61bc None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:33.883954 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 170/337] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:33.889931 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc2522-5efa-45cd-a1e3-65ebb7f83558 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:33.890428 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc2522-5efa-45cd-a1e3-65ebb7f83558 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:33.890688 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc2522-5efa-45cd-a1e3-65ebb7f83558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:33.920097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-75dc2522-5efa-45cd-a1e3-65ebb7f83558 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:33.921322 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-75dc2522-5efa-45cd-a1e3-65ebb7f83558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:33.996106 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:34.011173 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 168/338] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:34.121095 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f938a9-c759-41df-98ba-cf52648b404c None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:34.122315 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f938a9-c759-41df-98ba-cf52648b404c None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:34.122793 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f938a9-c759-41df-98ba-cf52648b404c None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:34.124865 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 171/339] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:34.130985 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c15e5-e5cf-411e-b167-d052f24f53db None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:34.131456 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c15e5-e5cf-411e-b167-d052f24f53db None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:34.131687 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c15e5-e5cf-411e-b167-d052f24f53db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:34.156127 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-812c15e5-e5cf-411e-b167-d052f24f53db None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:34.157334 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-812c15e5-e5cf-411e-b167-d052f24f53db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:34.237743 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c977b-fb3c-4b8e-9335-bef34810ea12 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:34.237743 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c977b-fb3c-4b8e-9335-bef34810ea12 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:34.237743 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c977b-fb3c-4b8e-9335-bef34810ea12 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:34.238468 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 172/340] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:34.249234 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 169/341] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:34.253544 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c308f-d818-424b-9a5f-4084404f1334 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:34.254059 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c308f-d818-424b-9a5f-4084404f1334 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:34.254562 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c308f-d818-424b-9a5f-4084404f1334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:34.279592 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-d58c308f-d818-424b-9a5f-4084404f1334 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:34.281096 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-d58c308f-d818-424b-9a5f-4084404f1334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:34.381572 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c15e5-e5cf-411e-b167-d052f24f53db None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:34.391124 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 173/342] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:34.509759 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c15e5-e5cf-411e-b167-d052f24f53db None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:34.539564 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:34.553379 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7036147-18b3-4cbe-b681-12bb39dbbd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:34.554046 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7036147-18b3-4cbe-b681-12bb39dbbd75 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:34.554328 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7036147-18b3-4cbe-b681-12bb39dbbd75 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:34.554599 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7036147-18b3-4cbe-b681-12bb39dbbd75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:34.569441 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7036147-18b3-4cbe-b681-12bb39dbbd75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:34.570888 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7036147-18b3-4cbe-b681-12bb39dbbd75 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:34.591424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-e7036147-18b3-4cbe-b681-12bb39dbbd75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:34.591424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:34.591424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:34.591424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:34.591424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:34.591424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:34.591424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:34.591424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:34.591424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:34.591424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:34.591424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:34.591424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:34.591424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:34.591424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:34.591424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:34.591424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:34.592408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:34.592408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:34.592408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:34.592408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:34.592408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:34.592408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:34.592408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:34.592408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:34.592408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:34.592408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:34.592408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:34.592408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:34.592408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:34.592408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:34.592408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:34.592408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:34.592408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:34.593273 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:34.593273 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:34.593273 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:34.593273 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 170/343] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:57:34.601197 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58c308f-d818-424b-9a5f-4084404f1334 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:34.615481 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7036147-18b3-4cbe-b681-12bb39dbbd75 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:34.717167 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7036147-18b3-4cbe-b681-12bb39dbbd75 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:34.754436 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f16aa4-6e79-460d-9097-f24484e8cdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:34.755083 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f16aa4-6e79-460d-9097-f24484e8cdd7 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:34.755376 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f16aa4-6e79-460d-9097-f24484e8cdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:34.755653 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f16aa4-6e79-460d-9097-f24484e8cdd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:34.767204 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f16aa4-6e79-460d-9097-f24484e8cdd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:34.768727 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f16aa4-6e79-460d-9097-f24484e8cdd7 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:34.783930 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-e2f16aa4-6e79-460d-9097-f24484e8cdd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:34.783930 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:34.783930 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:34.783930 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:34.783930 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:34.783930 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:34.783930 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:34.783930 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:34.783930 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:34.783930 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:34.783930 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:34.783930 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:34.783930 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:34.783930 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:34.783930 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:34.783930 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:34.784828 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:34.784828 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:34.784828 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:34.784828 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:34.784828 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:34.784828 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:34.784828 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:34.784828 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:34.784828 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:34.784828 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:34.784828 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:34.784828 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:34.784828 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:34.784828 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:34.784828 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:34.784828 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:34.784828 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:34.785673 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:34.785673 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:34.785673 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:34.785673 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 171/344] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:57:34.790853 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f16aa4-6e79-460d-9097-f24484e8cdd7 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:34.825527 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:34.838465 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:34.853760 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58c308f-d818-424b-9a5f-4084404f1334 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:34.918103 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f16aa4-6e79-460d-9097-f24484e8cdd7 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:34.920074 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149bdc23-2336-4fc9-9645-f3f5e5e9e734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:34.920692 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149bdc23-2336-4fc9-9645-f3f5e5e9e734 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:34.920974 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149bdc23-2336-4fc9-9645-f3f5e5e9e734 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:34.921246 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149bdc23-2336-4fc9-9645-f3f5e5e9e734 None admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:34.922113 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149bdc23-2336-4fc9-9645-f3f5e5e9e734 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:34.922422 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149bdc23-2336-4fc9-9645-f3f5e5e9e734 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:34.923837 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149bdc23-2336-4fc9-9645-f3f5e5e9e734 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:34.941252 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 174/345] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:57:34.946078 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149bdc23-2336-4fc9-9645-f3f5e5e9e734 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:34.963140 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35023f11-8af0-4e75-a786-8ac897df5adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:34.963750 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35023f11-8af0-4e75-a786-8ac897df5adf None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:34.964032 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35023f11-8af0-4e75-a786-8ac897df5adf None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:34.964306 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35023f11-8af0-4e75-a786-8ac897df5adf None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:34.964948 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35023f11-8af0-4e75-a786-8ac897df5adf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:34.965805 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35023f11-8af0-4e75-a786-8ac897df5adf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:34.967260 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35023f11-8af0-4e75-a786-8ac897df5adf None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:34.979097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 172/346] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:57:34.985322 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ae2be-38c9-42a7-8cf6-1534537a40c8 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:34.985614 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ae2be-38c9-42a7-8cf6-1534537a40c8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:34.985971 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ae2be-38c9-42a7-8cf6-1534537a40c8 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:34.987450 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 173/347] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:34.988123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ae2be-38c9-42a7-8cf6-1534537a40c8 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:35.060897 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149bdc23-2336-4fc9-9645-f3f5e5e9e734 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:35.078262 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:35.095904 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ae2be-38c9-42a7-8cf6-1534537a40c8 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:35.096283 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44571312-b59d-420f-9dbf-e2f00e7bfd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:35.096914 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44571312-b59d-420f-9dbf-e2f00e7bfd51 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:35.097191 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44571312-b59d-420f-9dbf-e2f00e7bfd51 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:35.097471 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44571312-b59d-420f-9dbf-e2f00e7bfd51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:35.108083 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44571312-b59d-420f-9dbf-e2f00e7bfd51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:35.109536 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44571312-b59d-420f-9dbf-e2f00e7bfd51 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:35.123831 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-44571312-b59d-420f-9dbf-e2f00e7bfd51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:35.123831 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:35.123831 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:35.123831 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:35.123831 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:35.123831 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:35.123831 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:35.123831 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:35.123831 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:35.123831 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:35.123831 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:35.123831 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:35.123831 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:35.123831 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:35.123831 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:35.123831 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:35.124726 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:35.124726 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:35.124726 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:35.124726 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:35.124726 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:35.124726 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:35.124726 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:35.124726 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:35.124726 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:35.124726 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:35.124726 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:35.124726 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:35.124726 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:35.124726 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:35.124726 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:35.124726 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:35.124726 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:35.125547 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:35.125547 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:35.125547 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:35.125547 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 175/348] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:35.125775 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44571312-b59d-420f-9dbf-e2f00e7bfd51 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:35.135279 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:35.135405 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84563fac-ca2b-4662-80c0-52079599855e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:35.135974 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84563fac-ca2b-4662-80c0-52079599855e None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:35.136254 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84563fac-ca2b-4662-80c0-52079599855e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:35.136527 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84563fac-ca2b-4662-80c0-52079599855e None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:35.137749 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84563fac-ca2b-4662-80c0-52079599855e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:35.139364 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84563fac-ca2b-4662-80c0-52079599855e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:35.167103 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 174/349] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 13 12:57:34 2022] POST /identity/v3/users => generated 246 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:57:35.167103 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a931da0-5a17-4df3-bc6c-6b7c1cf5e7d4 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:35.167103 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a931da0-5a17-4df3-bc6c-6b7c1cf5e7d4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:35.167103 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a931da0-5a17-4df3-bc6c-6b7c1cf5e7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:35.190596 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-2a931da0-5a17-4df3-bc6c-6b7c1cf5e7d4 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:35.191854 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2a931da0-5a17-4df3-bc6c-6b7c1cf5e7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:35.257233 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44571312-b59d-420f-9dbf-e2f00e7bfd51 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:35.285080 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 175/350] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:35.285955 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a931da0-5a17-4df3-bc6c-6b7c1cf5e7d4 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:35.294985 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21344637-710d-4dbf-8304-9061eabae068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:35.295837 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21344637-710d-4dbf-8304-9061eabae068 None admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:35.296196 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21344637-710d-4dbf-8304-9061eabae068 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:35.296586 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21344637-710d-4dbf-8304-9061eabae068 None admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:35.297388 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21344637-710d-4dbf-8304-9061eabae068 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:35.299000 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21344637-710d-4dbf-8304-9061eabae068 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:35.357663 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 176/351] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 13 12:57:35 2022] POST /identity/v3/users => generated 248 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:57:35.375546 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:35.416173 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a931da0-5a17-4df3-bc6c-6b7c1cf5e7d4 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:35.458451 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a7dc2-7988-4f4c-a93e-436e7f9b384b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:35.459102 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a7dc2-7988-4f4c-a93e-436e7f9b384b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:35.459377 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a7dc2-7988-4f4c-a93e-436e7f9b384b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:35.459644 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a7dc2-7988-4f4c-a93e-436e7f9b384b None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:35.460306 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a7dc2-7988-4f4c-a93e-436e7f9b384b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:35.460670 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a7dc2-7988-4f4c-a93e-436e7f9b384b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:35.462113 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a7dc2-7988-4f4c-a93e-436e7f9b384b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:35.475550 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 176/352] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:35.481254 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21344637-710d-4dbf-8304-9061eabae068 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:35.556660 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:35.597377 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21344637-710d-4dbf-8304-9061eabae068 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:35.598833 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a7dc2-7988-4f4c-a93e-436e7f9b384b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:35.638676 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4cab8-7342-4759-8f37-272d5a181c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:35.640291 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4cab8-7342-4759-8f37-272d5a181c68 None admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:35.640730 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4cab8-7342-4759-8f37-272d5a181c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:35.641175 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4cab8-7342-4759-8f37-272d5a181c68 None admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:35.642206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d4cab8-7342-4759-8f37-272d5a181c68 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:35.644669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d4cab8-7342-4759-8f37-272d5a181c68 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:35.678081 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 177/353] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 13 12:57:35 2022] POST /identity/v3/users => generated 248 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:57:35.739380 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a7dc2-7988-4f4c-a93e-436e7f9b384b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:35.764376 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c6e50-47c5-4bc7-8f41-e047fc04117a None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:35.764914 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c6e50-47c5-4bc7-8f41-e047fc04117a None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:35.765592 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c6e50-47c5-4bc7-8f41-e047fc04117a None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:35.768373 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 178/354] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 13 12:57:35.774888 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11b932-387b-41cc-bde5-97cd317cd5ff None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:35.776030 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11b932-387b-41cc-bde5-97cd317cd5ff None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:35.776467 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11b932-387b-41cc-bde5-97cd317cd5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:35.782355 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:35.792886 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516b973-8199-42de-9d9c-52690937f9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:35.794096 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516b973-8199-42de-9d9c-52690937f9e2 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:35.794353 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516b973-8199-42de-9d9c-52690937f9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:35.794625 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516b973-8199-42de-9d9c-52690937f9e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:35.809528 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-0d11b932-387b-41cc-bde5-97cd317cd5ff None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:35.810670 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-0d11b932-387b-41cc-bde5-97cd317cd5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:35.818807 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4516b973-8199-42de-9d9c-52690937f9e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:35.820346 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4516b973-8199-42de-9d9c-52690937f9e2 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:35.841757 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-4516b973-8199-42de-9d9c-52690937f9e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:35.841757 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:35.841757 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:35.841757 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:35.841757 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:35.841757 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:35.841757 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:35.841757 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:35.841757 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:35.841757 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:35.841757 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:35.841757 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:35.841757 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:35.841757 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:35.841757 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:35.841757 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:35.842909 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:35.842909 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:35.842909 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:35.842909 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:35.842909 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:35.842909 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:35.842909 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:35.842909 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:35.842909 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:35.842909 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:35.842909 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:35.842909 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:35.842909 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:35.842909 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:35.842909 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:35.842909 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:35.842909 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:35.843942 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:35.843942 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:35.843942 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:35.843942 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 177/355] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:35.850518 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516b973-8199-42de-9d9c-52690937f9e2 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:35.940205 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 179/356] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:35.984439 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516b973-8199-42de-9d9c-52690937f9e2 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:36.017762 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce0363-39f8-4bfc-b194-14667e1a04cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:36.018368 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce0363-39f8-4bfc-b194-14667e1a04cd None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:36.018648 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce0363-39f8-4bfc-b194-14667e1a04cd None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:36.018920 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce0363-39f8-4bfc-b194-14667e1a04cd None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:36.019649 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce0363-39f8-4bfc-b194-14667e1a04cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:36.019956 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce0363-39f8-4bfc-b194-14667e1a04cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:36.021443 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce0363-39f8-4bfc-b194-14667e1a04cd None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:36.034337 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 178/357] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:57:36.039687 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11b932-387b-41cc-bde5-97cd317cd5ff None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:36.092301 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:36.147540 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11b932-387b-41cc-bde5-97cd317cd5ff None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:36.193490 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e99b08-47f7-4a4b-830d-26da896e74bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:36.194477 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e99b08-47f7-4a4b-830d-26da896e74bc None admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:36.194950 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e99b08-47f7-4a4b-830d-26da896e74bc None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:36.195433 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e99b08-47f7-4a4b-830d-26da896e74bc None admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:36.196266 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e99b08-47f7-4a4b-830d-26da896e74bc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:36.197917 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e99b08-47f7-4a4b-830d-26da896e74bc None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:36.227995 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 180/358] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 13 12:57:36 2022] POST /identity/v3/users => generated 249 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:57:36.236654 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce0363-39f8-4bfc-b194-14667e1a04cd None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:36.286899 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:36.360587 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce0363-39f8-4bfc-b194-14667e1a04cd None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:36.417954 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff4a89-11b4-42be-b1e7-127d251aa0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:36.419591 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff4a89-11b4-42be-b1e7-127d251aa0a6 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:36.420085 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff4a89-11b4-42be-b1e7-127d251aa0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:36.420584 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff4a89-11b4-42be-b1e7-127d251aa0a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:36.421523 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff4a89-11b4-42be-b1e7-127d251aa0a6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:36.423226 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff4a89-11b4-42be-b1e7-127d251aa0a6 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:36.436621 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.sql.core [None req-e8ff4a89-11b4-42be-b1e7-127d251aa0a6 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 13 12:57:36.436621 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 13 12:57:36.436621 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [parameters: {'id': '32d352e8b7ed4690a886f80bcb1bf7a7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 13 12:57:36.436621 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81521) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 13 12:57:36.444276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-e8ff4a89-11b4-42be-b1e7-127d251aa0a6 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 13 12:57:36.444276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:36.444276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 13 12:57:36.444276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 13 12:57:36.444276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 13 12:57:36.444276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 13 12:57:36.444276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 13 12:57:36.444276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result = self._query(query) Oct 13 12:57:36.444276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 13 12:57:36.444276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application conn.query(q) Oct 13 12:57:36.444276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 13 12:57:36.444276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 13 12:57:36.444276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 13 12:57:36.444276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result.read() Oct 13 12:57:36.444276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 13 12:57:36.445221 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 13 12:57:36.445221 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 13 12:57:36.445221 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 13 12:57:36.445221 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 13 12:57:36.445221 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 13 12:57:36.445221 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 13 12:57:36.445221 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 13 12:57:36.445221 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 13 12:57:36.445221 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:36.445221 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 13 12:57:36.445221 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:36.445221 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:36.445221 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 13 12:57:36.445221 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 13 12:57:36.445221 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 13 12:57:36.445221 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return ref.to_dict() Oct 13 12:57:36.445221 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 13 12:57:36.445221 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application next(self.gen) Oct 13 12:57:36.446143 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 13 12:57:36.446143 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application yield resource Oct 13 12:57:36.446143 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 13 12:57:36.446143 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application next(self.gen) Oct 13 12:57:36.446143 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 13 12:57:36.446143 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.session.rollback() Oct 13 12:57:36.446143 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 13 12:57:36.446143 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.force_reraise() Oct 13 12:57:36.446143 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 13 12:57:36.446143 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise self.value Oct 13 12:57:36.446143 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 13 12:57:36.446143 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 13 12:57:36.446143 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 13 12:57:36.446143 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application session.commit() Oct 13 12:57:36.446143 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 13 12:57:36.446143 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 13 12:57:36.446143 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 13 12:57:36.446998 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._prepare_impl() Oct 13 12:57:36.446998 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 13 12:57:36.446998 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.session.flush() Oct 13 12:57:36.446998 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 13 12:57:36.446998 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._flush(objects) Oct 13 12:57:36.446998 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 13 12:57:36.446998 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 13 12:57:36.446998 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 13 12:57:36.446998 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application compat.raise_( Oct 13 12:57:36.446998 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 13 12:57:36.446998 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception Oct 13 12:57:36.446998 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 13 12:57:36.446998 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application flush_context.execute() Oct 13 12:57:36.446998 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 13 12:57:36.446998 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rec.execute(self) Oct 13 12:57:36.446998 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 13 12:57:36.446998 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 13 12:57:36.448605 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 13 12:57:36.448605 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 13 12:57:36.448605 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 13 12:57:36.448605 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 13 12:57:36.448605 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 13 12:57:36.448605 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 13 12:57:36.448605 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 13 12:57:36.448605 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 13 12:57:36.448605 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 13 12:57:36.448605 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 13 12:57:36.448605 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 13 12:57:36.448605 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 13 12:57:36.448605 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 13 12:57:36.448605 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 13 12:57:36.448605 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 13 12:57:36.448605 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception Oct 13 12:57:36.450656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 13 12:57:36.450656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 13 12:57:36.450656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 13 12:57:36.450656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 13 12:57:36.450656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 13 12:57:36.450656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result = self._query(query) Oct 13 12:57:36.450656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 13 12:57:36.450656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application conn.query(q) Oct 13 12:57:36.450656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 13 12:57:36.450656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 13 12:57:36.450656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 13 12:57:36.450656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result.read() Oct 13 12:57:36.450656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 13 12:57:36.450656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 13 12:57:36.450656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 13 12:57:36.450656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 13 12:57:36.452641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 13 12:57:36.452641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 13 12:57:36.452641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 13 12:57:36.452641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 13 12:57:36.452641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 13 12:57:36.452641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 13 12:57:36.452641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [parameters: {'id': '32d352e8b7ed4690a886f80bcb1bf7a7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 13 12:57:36.452641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 13 12:57:36.452641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:36.452641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 12:57:36.452641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:36.452641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:36.452641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:36.452641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:36.452641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:36.452641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:36.454704 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:36.454704 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:36.454704 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:36.454704 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:36.454704 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:36.454704 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:36.454704 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 13 12:57:36.454704 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 13 12:57:36.454704 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:36.454704 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:36.454704 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 13 12:57:36.454704 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 13 12:57:36.454704 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 13 12:57:36.454704 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 13 12:57:36.454704 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 13 12:57:36.454704 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:36.458637 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 179/359] 10.222.0.23 () {60 vars in 1131 bytes} [Thu Oct 13 12:57:36 2022] POST /identity/v3/roles => generated 144 bytes in 222 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:36.464383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e99b08-47f7-4a4b-830d-26da896e74bc None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:36.578612 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e99b08-47f7-4a4b-830d-26da896e74bc None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:36.581442 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:36.615753 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c8f3e-2d95-4273-b3ba-76a5be0b1b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:36.616315 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c8f3e-2d95-4273-b3ba-76a5be0b1b3d None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:36.616597 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c8f3e-2d95-4273-b3ba-76a5be0b1b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:36.616871 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c8f3e-2d95-4273-b3ba-76a5be0b1b3d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:36.627208 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77c8f3e-2d95-4273-b3ba-76a5be0b1b3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:36.632073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-d77c8f3e-2d95-4273-b3ba-76a5be0b1b3d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 13 12:57:36.632073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:36.632073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:36.632073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:36.632073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:36.632073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:36.632073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:36.632073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:36.632073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:36.632073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:36.632073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:36.632073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:36.632073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:36.632073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:36.632073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:36.632073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:36.632949 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:36.632949 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:36.632949 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:36.632949 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:36.632949 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:36.632949 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:36.632949 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:36.632949 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:36.632949 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:36.632949 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:36.632949 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:36.632949 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:36.632949 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:36.632949 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:36.632949 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:36.632949 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:36.633782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:36.633782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:36.633782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:36.633782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:36.633782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:36.633782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:36.633782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:36.633782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:36.633782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:36.633782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:36.633782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 13 12:57:36.633782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:36.633782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 181/360] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Oct 13 12:57:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:36.639092 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff4a89-11b4-42be-b1e7-127d251aa0a6 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:36.742585 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff4a89-11b4-42be-b1e7-127d251aa0a6 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:36.752411 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:36.778532 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca30033-47e1-4fd4-97a2-a0f22bed80bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:36.779176 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca30033-47e1-4fd4-97a2-a0f22bed80bd None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:36.779461 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca30033-47e1-4fd4-97a2-a0f22bed80bd None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:36.779734 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca30033-47e1-4fd4-97a2-a0f22bed80bd None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:36.780404 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca30033-47e1-4fd4-97a2-a0f22bed80bd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:36.780761 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca30033-47e1-4fd4-97a2-a0f22bed80bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:36.782131 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca30033-47e1-4fd4-97a2-a0f22bed80bd None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:36.790424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 180/361] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:57:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:37.949066 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8017305-c92d-4e42-8e51-6c9480ad9ae5 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:37.949640 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8017305-c92d-4e42-8e51-6c9480ad9ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:37.949722 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8017305-c92d-4e42-8e51-6c9480ad9ae5 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:37.951030 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 182/362] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:37.956663 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc1bdb-9323-48e2-ad2f-cff24b57604d None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:37.957087 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc1bdb-9323-48e2-ad2f-cff24b57604d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:37.957256 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc1bdb-9323-48e2-ad2f-cff24b57604d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:37.984083 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-dadc1bdb-9323-48e2-ad2f-cff24b57604d None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:37.985296 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-dadc1bdb-9323-48e2-ad2f-cff24b57604d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:38.058176 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 181/363] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:38.213271 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe97278-1458-4ea1-9925-85a392f94c48 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:38.213614 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe97278-1458-4ea1-9925-85a392f94c48 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:38.213785 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe97278-1458-4ea1-9925-85a392f94c48 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:38.215694 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 183/364] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:38.221593 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71872b-821f-4e18-a53a-881d1d9188bb None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:38.221874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71872b-821f-4e18-a53a-881d1d9188bb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:38.222143 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71872b-821f-4e18-a53a-881d1d9188bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:38.247993 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-5c71872b-821f-4e18-a53a-881d1d9188bb None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:38.249196 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5c71872b-821f-4e18-a53a-881d1d9188bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:38.321269 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 182/365] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:38.408129 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe97278-1458-4ea1-9925-85a392f94c48 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:38.540048 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe97278-1458-4ea1-9925-85a392f94c48 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:38.583620 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59e655-1ab2-489c-99aa-adb4e81e8230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:38.584603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59e655-1ab2-489c-99aa-adb4e81e8230 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:38.585098 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59e655-1ab2-489c-99aa-adb4e81e8230 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:38.585555 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59e655-1ab2-489c-99aa-adb4e81e8230 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:38.601292 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a59e655-1ab2-489c-99aa-adb4e81e8230 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:38.604559 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ef1f9-5ec2-41ab-a928-e1ffb4b256c7 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:38.604848 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ef1f9-5ec2-41ab-a928-e1ffb4b256c7 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:38.605124 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ef1f9-5ec2-41ab-a928-e1ffb4b256c7 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:38.606557 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 183/366] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 13 12:57:38.607302 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-6a59e655-1ab2-489c-99aa-adb4e81e8230 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 13 12:57:38.607302 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:38.607302 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:38.607302 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:38.607302 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:38.607302 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:38.607302 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:38.607302 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:38.607302 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:38.607302 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:38.607302 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:38.607302 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:38.607302 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:38.607302 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:38.607302 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:38.607302 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:38.608160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:38.608160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:38.608160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:38.608160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:38.608160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:38.608160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:38.608160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:38.608160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:38.608160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:38.608160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:38.608160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:38.608160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:38.608160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:38.608160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:38.608160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:38.608160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:38.608928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:38.608928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:38.608928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:38.608928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:38.608928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:38.608928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:38.608928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:38.608928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:38.608928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:38.608928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:38.608928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 13 12:57:38.608928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:38.611330 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 184/367] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Oct 13 12:57:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:38.611443 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f535ce-1ac6-46e7-8999-7ca3ded52794 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:38.611620 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f535ce-1ac6-46e7-8999-7ca3ded52794 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:38.614919 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f535ce-1ac6-46e7-8999-7ca3ded52794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:38.617528 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59e655-1ab2-489c-99aa-adb4e81e8230 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:38.647486 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:38.649415 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-03f535ce-1ac6-46e7-8999-7ca3ded52794 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:38.650631 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-03f535ce-1ac6-46e7-8999-7ca3ded52794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:38.685919 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:38.736341 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 184/368] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 13 12:57:38.737553 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f535ce-1ac6-46e7-8999-7ca3ded52794 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:38.745302 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59e655-1ab2-489c-99aa-adb4e81e8230 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:38.788771 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab7c97-88e3-4a6f-9df0-9830f6680537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:38.789322 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab7c97-88e3-4a6f-9df0-9830f6680537 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:38.789603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab7c97-88e3-4a6f-9df0-9830f6680537 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:38.789878 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab7c97-88e3-4a6f-9df0-9830f6680537 None admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:38.790898 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ab7c97-88e3-4a6f-9df0-9830f6680537 None admin] RBAC: Adding query filter params (name=service) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:38.790898 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ab7c97-88e3-4a6f-9df0-9830f6680537 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:38.792246 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ab7c97-88e3-4a6f-9df0-9830f6680537 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:38.801598 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 185/369] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Oct 13 12:57:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:57:38.808487 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab7c97-88e3-4a6f-9df0-9830f6680537 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:38.894606 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f535ce-1ac6-46e7-8999-7ca3ded52794 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:38.954163 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd820c94-c9f7-4f9c-81eb-e83dc45b25e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:38.954797 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd820c94-c9f7-4f9c-81eb-e83dc45b25e1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:38.955088 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd820c94-c9f7-4f9c-81eb-e83dc45b25e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:38.955364 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd820c94-c9f7-4f9c-81eb-e83dc45b25e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:38.975702 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd820c94-c9f7-4f9c-81eb-e83dc45b25e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:38.984401 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-fd820c94-c9f7-4f9c-81eb-e83dc45b25e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 13 12:57:38.984401 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:38.984401 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:38.984401 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:38.984401 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:38.984401 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:38.984401 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:38.984401 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:38.984401 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:38.984401 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:38.984401 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:38.984401 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:38.984401 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:38.984401 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:38.984401 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:38.984401 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:38.985239 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:38.985239 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:38.985239 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:38.985239 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:38.985239 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:38.985239 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:38.985239 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:38.985239 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:38.985239 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:38.985239 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:38.985239 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:38.985239 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:38.985239 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:38.985239 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:38.985239 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:38.985239 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:38.986031 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:38.986031 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:38.986031 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:38.986031 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:38.986031 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:38.986031 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:38.986031 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:38.986031 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:38.986031 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:38.986031 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:38.986031 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 13 12:57:38.986031 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:38.986031 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 185/370] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Oct 13 12:57:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:38.990465 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:38.995994 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd820c94-c9f7-4f9c-81eb-e83dc45b25e1 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:39.025017 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab7c97-88e3-4a6f-9df0-9830f6680537 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:39.049893 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:39.094336 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229eb8e-f773-4485-b9a4-f26c65f96254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:39.094983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229eb8e-f773-4485-b9a4-f26c65f96254 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:39.095551 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229eb8e-f773-4485-b9a4-f26c65f96254 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:39.096003 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4229eb8e-f773-4485-b9a4-f26c65f96254 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:39.111610 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4229eb8e-f773-4485-b9a4-f26c65f96254 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:39.113070 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4229eb8e-f773-4485-b9a4-f26c65f96254 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:39.133855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-4229eb8e-f773-4485-b9a4-f26c65f96254 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:39.133855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:39.133855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:39.133855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:39.133855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:39.133855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:39.133855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:39.133855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:39.133855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:39.133855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:39.133855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:39.133855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:39.133855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:39.133855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:39.133855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:39.133855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:39.135634 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:39.135634 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:39.135634 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:39.135634 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:39.135634 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:39.135634 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:39.135634 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:39.135634 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:39.135634 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:39.135634 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:39.135634 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:39.135634 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:39.135634 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:39.135634 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:39.135634 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:39.135634 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:39.135634 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:39.137699 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:39.137699 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:39.137699 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:39.137699 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 186/371] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:39.137699 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229eb8e-f773-4485-b9a4-f26c65f96254 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:39.158564 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd820c94-c9f7-4f9c-81eb-e83dc45b25e1 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:39.194225 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9876224-b22d-4bcf-8b7c-9fb1cd85fadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:39.194837 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9876224-b22d-4bcf-8b7c-9fb1cd85fadb None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:39.195147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9876224-b22d-4bcf-8b7c-9fb1cd85fadb None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:39.195425 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9876224-b22d-4bcf-8b7c-9fb1cd85fadb None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:39.196192 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9876224-b22d-4bcf-8b7c-9fb1cd85fadb None admin] RBAC: Adding query filter params (name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:39.196503 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9876224-b22d-4bcf-8b7c-9fb1cd85fadb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:39.197871 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9876224-b22d-4bcf-8b7c-9fb1cd85fadb None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:39.207813 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 186/372] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Oct 13 12:57:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:39.211933 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9876224-b22d-4bcf-8b7c-9fb1cd85fadb None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:39.252669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4229eb8e-f773-4485-b9a4-f26c65f96254 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:39.288315 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2b1a7d-1e5f-4a2a-b6e1-8d9e83ea422d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:39.288947 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b1a7d-1e5f-4a2a-b6e1-8d9e83ea422d None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:39.289226 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b1a7d-1e5f-4a2a-b6e1-8d9e83ea422d None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:39.289499 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b1a7d-1e5f-4a2a-b6e1-8d9e83ea422d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:39.297053 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:39.301448 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2b1a7d-1e5f-4a2a-b6e1-8d9e83ea422d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:39.303108 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:39.306842 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-2b2b1a7d-1e5f-4a2a-b6e1-8d9e83ea422d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 13 12:57:39.306842 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:39.306842 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:39.306842 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:39.306842 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:39.306842 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:39.306842 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:39.306842 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:39.306842 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:39.306842 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:39.306842 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:39.306842 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:39.306842 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:39.306842 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:39.306842 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:39.306842 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:39.307776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:39.307776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:39.307776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:39.307776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:39.307776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:39.307776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:39.307776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:39.307776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:39.307776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:39.307776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:39.307776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:39.307776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:39.307776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:39.307776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:39.307776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:39.307776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:39.308563 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:39.308563 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:39.308563 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:39.308563 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:39.308563 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:39.308563 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:39.308563 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:39.308563 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:39.308563 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:39.308563 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:39.308563 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 13 12:57:39.308563 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:39.308563 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 187/373] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Oct 13 12:57:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:39.310172 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387740e9-3b5d-4fa3-a828-28847f86e789 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:39.310172 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387740e9-3b5d-4fa3-a828-28847f86e789 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:39.310297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387740e9-3b5d-4fa3-a828-28847f86e789 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:39.311465 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 188/374] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:39.316350 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387740e9-3b5d-4fa3-a828-28847f86e789 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:39.329627 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9876224-b22d-4bcf-8b7c-9fb1cd85fadb None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:39.364393 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921078c7-5e8b-434f-913d-1ba2d73d00fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:39.365281 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921078c7-5e8b-434f-913d-1ba2d73d00fc None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:39.365738 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921078c7-5e8b-434f-913d-1ba2d73d00fc None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:39.366156 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921078c7-5e8b-434f-913d-1ba2d73d00fc None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:39.366952 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921078c7-5e8b-434f-913d-1ba2d73d00fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:39.367291 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921078c7-5e8b-434f-913d-1ba2d73d00fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:39.368657 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921078c7-5e8b-434f-913d-1ba2d73d00fc None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:39.385963 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 187/375] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:39.385963 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921078c7-5e8b-434f-913d-1ba2d73d00fc None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:39.427766 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387740e9-3b5d-4fa3-a828-28847f86e789 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:39.461368 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eab1ee-c6b4-43ab-aaef-517bd65be843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:39.461999 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eab1ee-c6b4-43ab-aaef-517bd65be843 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:39.462270 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eab1ee-c6b4-43ab-aaef-517bd65be843 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:39.462546 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eab1ee-c6b4-43ab-aaef-517bd65be843 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:39.473354 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eab1ee-c6b4-43ab-aaef-517bd65be843 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:39.474812 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eab1ee-c6b4-43ab-aaef-517bd65be843 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:39.489174 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921078c7-5e8b-434f-913d-1ba2d73d00fc None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:39.490281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-d3eab1ee-c6b4-43ab-aaef-517bd65be843 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:39.490281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:39.490281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:39.490281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:39.490281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:39.490281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:39.490281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:39.490281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:39.490281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:39.490281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:39.490281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:39.490281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:39.490281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:39.490281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:39.490281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:39.490281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:39.491271 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:39.491271 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:39.491271 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:39.491271 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:39.491271 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:39.491271 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:39.491271 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:39.491271 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:39.491271 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:39.491271 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:39.491271 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:39.491271 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:39.491271 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:39.491271 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:39.491271 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:39.491271 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:39.491271 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:39.497093 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:39.497093 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:39.497093 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:39.497093 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 189/376] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:39.497093 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f03ec-1515-450e-9cfc-97094c5060d1 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:39.497093 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f03ec-1515-450e-9cfc-97094c5060d1 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:39.497093 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f03ec-1515-450e-9cfc-97094c5060d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:39.510775 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-a03f03ec-1515-450e-9cfc-97094c5060d1 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:39.512520 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-a03f03ec-1515-450e-9cfc-97094c5060d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:39.530326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe3e414-ae1d-49e7-b2bb-cb6951a024da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:39.530935 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3e414-ae1d-49e7-b2bb-cb6951a024da None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:39.531248 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3e414-ae1d-49e7-b2bb-cb6951a024da None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:39.531523 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3e414-ae1d-49e7-b2bb-cb6951a024da None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:39.532181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe3e414-ae1d-49e7-b2bb-cb6951a024da None admin] RBAC: Adding query filter params (name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:39.532484 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe3e414-ae1d-49e7-b2bb-cb6951a024da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:39.533834 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe3e414-ae1d-49e7-b2bb-cb6951a024da None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:39.543321 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 188/377] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Oct 13 12:57:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:39.548830 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe3e414-ae1d-49e7-b2bb-cb6951a024da None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:39.552991 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:39.566439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:39.599081 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 190/378] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:39.599081 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f03ec-1515-450e-9cfc-97094c5060d1 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:39.669166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe3e414-ae1d-49e7-b2bb-cb6951a024da None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:39.706308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23b6cf-0b2d-413d-a762-486200ed8eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:39.707263 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23b6cf-0b2d-413d-a762-486200ed8eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:39.707543 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23b6cf-0b2d-413d-a762-486200ed8eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:39.707873 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23b6cf-0b2d-413d-a762-486200ed8eb2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:39.709397 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f03ec-1515-450e-9cfc-97094c5060d1 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:39.716245 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b23b6cf-0b2d-413d-a762-486200ed8eb2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:39.718094 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b23b6cf-0b2d-413d-a762-486200ed8eb2 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:39.733293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-4b23b6cf-0b2d-413d-a762-486200ed8eb2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 13 12:57:39.733293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:39.733293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:39.733293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:39.733293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:39.733293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:39.733293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:39.733293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:39.733293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:39.733293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:39.733293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:39.733293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:39.733293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 13 12:57:39.733293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 13 12:57:39.733293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 13 12:57:39.733293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 13 12:57:39.734184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:39.734184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:39.734184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 13 12:57:39.734184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:57:39.734184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 13 12:57:39.734184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:57:39.734184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:39.734184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:39.734184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:39.734184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:39.734184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:39.734184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:39.734184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:39.734184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:39.734184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:39.734184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:39.734184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:39.734184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:39.735093 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:39.735093 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:39.735093 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 13 12:57:39.735093 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 13 12:57:39.735093 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 13 12:57:39.735093 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 13 12:57:39.735093 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 13 12:57:39.735093 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 13 12:57:39.735093 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 13 12:57:39.735093 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:39.736299 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 189/379] 10.222.0.23 () {56 vars in 1093 bytes} [Thu Oct 13 12:57:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:39.739929 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23b6cf-0b2d-413d-a762-486200ed8eb2 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:39.750232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6668007e-9b25-463c-9558-a59b7896c4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:39.750848 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668007e-9b25-463c-9558-a59b7896c4d9 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:39.751162 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668007e-9b25-463c-9558-a59b7896c4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:39.751438 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668007e-9b25-463c-9558-a59b7896c4d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:39.752087 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6668007e-9b25-463c-9558-a59b7896c4d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:39.752391 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6668007e-9b25-463c-9558-a59b7896c4d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:39.753751 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6668007e-9b25-463c-9558-a59b7896c4d9 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:39.769679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 191/380] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:39.769679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce6b8e-e9d9-4028-ab5d-fda268f83145 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:39.769679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce6b8e-e9d9-4028-ab5d-fda268f83145 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:39.769679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce6b8e-e9d9-4028-ab5d-fda268f83145 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:39.772508 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 192/381] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:39.773280 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce6b8e-e9d9-4028-ab5d-fda268f83145 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:39.810938 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:39.816267 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:39.854920 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23b6cf-0b2d-413d-a762-486200ed8eb2 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:39.877762 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce6b8e-e9d9-4028-ab5d-fda268f83145 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:39.890105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86037a86-4284-4bf7-a440-aa4924d68594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:39.890726 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86037a86-4284-4bf7-a440-aa4924d68594 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:39.891008 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86037a86-4284-4bf7-a440-aa4924d68594 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:39.891309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86037a86-4284-4bf7-a440-aa4924d68594 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:39.901666 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86037a86-4284-4bf7-a440-aa4924d68594 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:39.903198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86037a86-4284-4bf7-a440-aa4924d68594 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:39.911020 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5faf59c-3cb7-4cba-9e1f-43465355aa75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:39.911650 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5faf59c-3cb7-4cba-9e1f-43465355aa75 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:39.911932 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5faf59c-3cb7-4cba-9e1f-43465355aa75 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:39.912202 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5faf59c-3cb7-4cba-9e1f-43465355aa75 None admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:39.912928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5faf59c-3cb7-4cba-9e1f-43465355aa75 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:39.913233 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5faf59c-3cb7-4cba-9e1f-43465355aa75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:39.914655 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5faf59c-3cb7-4cba-9e1f-43465355aa75 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:39.917747 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-86037a86-4284-4bf7-a440-aa4924d68594 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:39.917747 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:39.917747 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:39.917747 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:39.917747 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:39.917747 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:39.917747 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:39.917747 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:39.917747 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:39.917747 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:39.917747 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:39.917747 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:39.917747 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:39.917747 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:39.917747 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:39.917747 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:39.918815 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:39.918815 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:39.918815 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:39.918815 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:39.918815 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:39.918815 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:39.918815 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:39.918815 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:39.918815 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:39.918815 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:39.918815 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:39.918815 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:39.918815 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:39.918815 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:39.918815 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:39.918815 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:39.918815 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:39.919688 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:39.919688 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:39.919688 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:39.919688 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 190/382] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:39.919688 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86037a86-4284-4bf7-a440-aa4924d68594 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:39.952434 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 193/383] 10.222.0.23 () {56 vars in 1133 bytes} [Thu Oct 13 12:57:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:39.953399 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b097c59-82dc-4d57-8bcc-36ffc92f250b None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:39.953685 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b097c59-82dc-4d57-8bcc-36ffc92f250b None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:39.953962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b097c59-82dc-4d57-8bcc-36ffc92f250b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:39.971086 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-9b097c59-82dc-4d57-8bcc-36ffc92f250b None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:39.972296 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-9b097c59-82dc-4d57-8bcc-36ffc92f250b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:40.027730 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86037a86-4284-4bf7-a440-aa4924d68594 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:40.036024 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 194/384] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:40.036828 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b097c59-82dc-4d57-8bcc-36ffc92f250b None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:40.055802 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:40.065020 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:40.079671 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676043a-8f6d-4b7a-9dcb-ed6dc3a1bed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:40.080677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676043a-8f6d-4b7a-9dcb-ed6dc3a1bed8 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:40.080957 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676043a-8f6d-4b7a-9dcb-ed6dc3a1bed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:40.081225 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676043a-8f6d-4b7a-9dcb-ed6dc3a1bed8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:40.090051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2676043a-8f6d-4b7a-9dcb-ed6dc3a1bed8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:40.092274 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2676043a-8f6d-4b7a-9dcb-ed6dc3a1bed8 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:40.106006 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-2676043a-8f6d-4b7a-9dcb-ed6dc3a1bed8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 13 12:57:40.106006 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:40.106006 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:40.106006 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:40.106006 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:40.106006 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:40.106006 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:40.106006 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:40.106006 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:40.106006 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:40.106006 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:40.106006 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:40.106006 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 13 12:57:40.106006 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 13 12:57:40.106006 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 13 12:57:40.106006 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 13 12:57:40.106933 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:40.106933 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:40.106933 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 13 12:57:40.106933 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:57:40.106933 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 13 12:57:40.106933 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:57:40.106933 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:40.106933 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:40.106933 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:40.106933 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:40.106933 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:40.106933 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:40.106933 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:40.106933 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:40.106933 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:40.106933 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:40.106933 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:40.106933 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:40.107845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:40.107845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:40.107845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 13 12:57:40.107845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 13 12:57:40.107845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 13 12:57:40.107845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 13 12:57:40.107845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 13 12:57:40.107845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 13 12:57:40.107845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 13 12:57:40.107845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:40.107845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 191/385] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:57:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:40.107845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676043a-8f6d-4b7a-9dcb-ed6dc3a1bed8 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:40.143244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b097c59-82dc-4d57-8bcc-36ffc92f250b None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:40.177451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d386ed-d1e7-4de2-a78c-46b6381b085e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:40.178191 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d386ed-d1e7-4de2-a78c-46b6381b085e None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:40.178469 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d386ed-d1e7-4de2-a78c-46b6381b085e None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:40.178735 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d386ed-d1e7-4de2-a78c-46b6381b085e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:40.189897 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d386ed-d1e7-4de2-a78c-46b6381b085e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:40.194613 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-91d386ed-d1e7-4de2-a78c-46b6381b085e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 13 12:57:40.194613 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:40.194613 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:40.194613 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:40.194613 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:40.194613 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:40.194613 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:40.194613 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:40.194613 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:40.194613 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:40.194613 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:40.194613 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:40.194613 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:40.194613 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:40.194613 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:40.194613 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:40.195656 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:40.195656 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:40.195656 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:40.195656 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:40.195656 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:40.195656 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:40.195656 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:40.195656 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:40.195656 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:40.195656 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:40.195656 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:40.195656 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:40.195656 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:40.195656 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:40.195656 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:40.195656 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:40.196460 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:40.196460 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:40.196460 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:40.196460 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:40.196460 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:40.196460 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:40.196460 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:40.196460 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:40.196460 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:40.196460 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:40.196460 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 13 12:57:40.196460 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:40.196460 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 195/386] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Oct 13 12:57:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:40.196460 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d386ed-d1e7-4de2-a78c-46b6381b085e None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:40.213380 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2676043a-8f6d-4b7a-9dcb-ed6dc3a1bed8 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:40.248373 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde8ee9-84d0-4dce-a985-58fd7ef149e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:40.248855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde8ee9-84d0-4dce-a985-58fd7ef149e8 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:40.249131 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde8ee9-84d0-4dce-a985-58fd7ef149e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:40.249399 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde8ee9-84d0-4dce-a985-58fd7ef149e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:40.250027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde8ee9-84d0-4dce-a985-58fd7ef149e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:40.250323 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde8ee9-84d0-4dce-a985-58fd7ef149e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:40.251697 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde8ee9-84d0-4dce-a985-58fd7ef149e8 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:40.264348 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 192/387] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:40.264952 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde8ee9-84d0-4dce-a985-58fd7ef149e8 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:40.298329 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:40.300580 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:40.312521 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d386ed-d1e7-4de2-a78c-46b6381b085e None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:40.345325 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea34c45-2469-47e1-b927-953a40d88482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:40.345887 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea34c45-2469-47e1-b927-953a40d88482 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:40.346177 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea34c45-2469-47e1-b927-953a40d88482 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:40.346446 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea34c45-2469-47e1-b927-953a40d88482 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:40.356770 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea34c45-2469-47e1-b927-953a40d88482 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:40.358227 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea34c45-2469-47e1-b927-953a40d88482 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:40.368309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde8ee9-84d0-4dce-a985-58fd7ef149e8 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:40.372439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-aea34c45-2469-47e1-b927-953a40d88482 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:40.372439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:40.372439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:40.372439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:40.372439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:40.372439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:40.372439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:40.372439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:40.372439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:40.372439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:40.372439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:40.372439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:40.372439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:40.372439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:40.372439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:40.372439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:40.373476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:40.373476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:40.373476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:40.373476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:40.373476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:40.373476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:40.373476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:40.373476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:40.373476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:40.373476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:40.373476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:40.373476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:40.373476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:40.373476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:40.373476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:40.373476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:40.373476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:40.378978 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:40.378978 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:40.378978 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:40.378978 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 196/388] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:40.378978 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea34c45-2469-47e1-b927-953a40d88482 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:40.401681 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be185853-5440-4919-9a7b-36b432203d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:40.402794 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be185853-5440-4919-9a7b-36b432203d6b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:40.403117 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be185853-5440-4919-9a7b-36b432203d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:40.403390 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be185853-5440-4919-9a7b-36b432203d6b None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:40.404113 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be185853-5440-4919-9a7b-36b432203d6b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:40.404415 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be185853-5440-4919-9a7b-36b432203d6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:40.405817 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be185853-5440-4919-9a7b-36b432203d6b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:40.441533 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 193/389] 10.222.0.23 () {56 vars in 1137 bytes} [Thu Oct 13 12:57:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:40.442330 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be185853-5440-4919-9a7b-36b432203d6b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:40.477956 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea34c45-2469-47e1-b927-953a40d88482 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:40.510098 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05969ab0-849f-46b8-9a30-1ba4ff13268c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:40.510702 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05969ab0-849f-46b8-9a30-1ba4ff13268c None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:40.510979 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05969ab0-849f-46b8-9a30-1ba4ff13268c None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:40.511279 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05969ab0-849f-46b8-9a30-1ba4ff13268c None admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:40.511990 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05969ab0-849f-46b8-9a30-1ba4ff13268c None admin] RBAC: Adding query filter params (name=service) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:40.512291 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05969ab0-849f-46b8-9a30-1ba4ff13268c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:40.513670 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05969ab0-849f-46b8-9a30-1ba4ff13268c None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:40.521521 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 197/390] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Oct 13 12:57:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:40.522296 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05969ab0-849f-46b8-9a30-1ba4ff13268c None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:40.524692 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:40.533157 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:40.540345 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be185853-5440-4919-9a7b-36b432203d6b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:40.574027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc5cf2-aece-49b3-8cb5-b628d2120a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:40.574666 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc5cf2-aece-49b3-8cb5-b628d2120a53 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:40.574943 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc5cf2-aece-49b3-8cb5-b628d2120a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:40.575241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc5cf2-aece-49b3-8cb5-b628d2120a53 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:40.582295 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc5cf2-aece-49b3-8cb5-b628d2120a53 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:40.583768 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc5cf2-aece-49b3-8cb5-b628d2120a53 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:40.595470 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-f9dc5cf2-aece-49b3-8cb5-b628d2120a53 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 13 12:57:40.595470 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:40.595470 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:40.595470 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:40.595470 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:40.595470 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:40.595470 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:40.595470 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:40.595470 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:40.595470 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:40.595470 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:40.595470 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:40.595470 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 13 12:57:40.595470 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 13 12:57:40.595470 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 13 12:57:40.595470 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 13 12:57:40.596397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:40.596397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:40.596397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 13 12:57:40.596397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:57:40.596397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 13 12:57:40.596397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:57:40.596397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:40.596397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:40.596397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:40.596397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:40.596397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:40.596397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:40.596397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:40.596397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:40.596397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:40.596397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:40.596397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:40.596397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:40.597284 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:40.597284 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:40.597284 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 13 12:57:40.597284 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 13 12:57:40.597284 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 13 12:57:40.597284 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 13 12:57:40.597284 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 13 12:57:40.597284 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 13 12:57:40.597284 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 13 12:57:40.597284 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:40.597284 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 194/391] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:57:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:40.597284 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc5cf2-aece-49b3-8cb5-b628d2120a53 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:40.622435 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05969ab0-849f-46b8-9a30-1ba4ff13268c None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:40.655745 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe668e-3176-4215-b5c1-c80d795b2aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:40.656462 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe668e-3176-4215-b5c1-c80d795b2aaa None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:40.656812 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe668e-3176-4215-b5c1-c80d795b2aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:40.657094 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe668e-3176-4215-b5c1-c80d795b2aaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:40.667190 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe668e-3176-4215-b5c1-c80d795b2aaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:40.669185 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe668e-3176-4215-b5c1-c80d795b2aaa None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:40.682754 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-bebe668e-3176-4215-b5c1-c80d795b2aaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:40.682754 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:40.682754 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:40.682754 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:40.682754 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:40.682754 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:40.682754 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:40.682754 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:40.682754 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:40.682754 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:40.682754 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:40.682754 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:40.682754 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:40.682754 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:40.682754 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:40.682754 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:40.683684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:40.683684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:40.683684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:40.683684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:40.683684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:40.683684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:40.683684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:40.683684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:40.683684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:40.683684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:40.683684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:40.683684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:40.683684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:40.683684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:40.683684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:40.683684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:40.683684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:40.684517 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:40.684517 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:40.684517 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:40.684517 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 198/392] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:40.684723 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe668e-3176-4215-b5c1-c80d795b2aaa None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:40.696946 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc5cf2-aece-49b3-8cb5-b628d2120a53 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:40.729745 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49662c-f9ba-41fb-aae5-98b1c575d6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:40.730336 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49662c-f9ba-41fb-aae5-98b1c575d6f4 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:40.730640 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49662c-f9ba-41fb-aae5-98b1c575d6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:40.730912 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49662c-f9ba-41fb-aae5-98b1c575d6f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:40.731621 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b49662c-f9ba-41fb-aae5-98b1c575d6f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:40.731923 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b49662c-f9ba-41fb-aae5-98b1c575d6f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:40.733299 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b49662c-f9ba-41fb-aae5-98b1c575d6f4 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:40.740232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:40.744789 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 195/393] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:40.745632 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49662c-f9ba-41fb-aae5-98b1c575d6f4 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:40.759673 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:40.787807 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe668e-3176-4215-b5c1-c80d795b2aaa None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:40.820127 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a33667-9923-4c21-bc97-caaea774d9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:40.820777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a33667-9923-4c21-bc97-caaea774d9ed None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:40.821058 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a33667-9923-4c21-bc97-caaea774d9ed None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:40.821331 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a33667-9923-4c21-bc97-caaea774d9ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:40.831742 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a33667-9923-4c21-bc97-caaea774d9ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:40.833193 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a33667-9923-4c21-bc97-caaea774d9ed None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:40.842864 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49662c-f9ba-41fb-aae5-98b1c575d6f4 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:40.848032 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-c5a33667-9923-4c21-bc97-caaea774d9ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:40.848032 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:40.848032 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:40.848032 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:40.848032 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:40.848032 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:40.848032 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:40.848032 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:40.848032 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:40.848032 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:40.848032 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:40.848032 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:40.848032 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:40.848032 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:40.848032 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:40.848032 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:40.849065 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:40.849065 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:40.849065 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:40.849065 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:40.849065 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:40.849065 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:40.849065 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:40.849065 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:40.849065 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:40.849065 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:40.849065 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:40.849065 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:40.849065 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:40.849065 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:40.849065 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:40.849065 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:40.849065 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:40.850201 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:40.850201 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:40.850201 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:40.850201 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 199/394] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:40.850201 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a33667-9923-4c21-bc97-caaea774d9ed None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:40.879171 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e2d75-d804-4f4c-9336-d0dbc3dc323b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:40.879926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e2d75-d804-4f4c-9336-d0dbc3dc323b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:40.880205 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e2d75-d804-4f4c-9336-d0dbc3dc323b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:40.880470 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e2d75-d804-4f4c-9336-d0dbc3dc323b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:40.891537 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e2d75-d804-4f4c-9336-d0dbc3dc323b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:40.892960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e2d75-d804-4f4c-9336-d0dbc3dc323b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:40.908109 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-673e2d75-d804-4f4c-9336-d0dbc3dc323b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:40.908109 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:40.908109 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:40.908109 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:40.908109 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:40.908109 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:40.908109 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:40.908109 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:40.908109 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:40.908109 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:40.908109 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:40.908109 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:40.908109 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:40.908109 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:40.908109 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:40.908109 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:40.909174 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:40.909174 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:40.909174 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:40.909174 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:40.909174 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:40.909174 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:40.909174 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:40.909174 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:40.909174 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:40.909174 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:40.909174 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:40.909174 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:40.909174 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:40.909174 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:40.909174 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:40.909174 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:40.909174 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:40.914922 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:40.914922 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:40.914922 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:40.914922 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 196/395] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:40.914922 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e2d75-d804-4f4c-9336-d0dbc3dc323b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:40.950409 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a33667-9923-4c21-bc97-caaea774d9ed None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:40.957331 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:40.980322 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:40.984140 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043eb71-5c32-4bf7-9df6-2a691d587769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:40.984740 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043eb71-5c32-4bf7-9df6-2a691d587769 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:40.985024 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043eb71-5c32-4bf7-9df6-2a691d587769 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:40.985294 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043eb71-5c32-4bf7-9df6-2a691d587769 None admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:40.986012 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043eb71-5c32-4bf7-9df6-2a691d587769 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:40.986350 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043eb71-5c32-4bf7-9df6-2a691d587769 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:40.987784 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043eb71-5c32-4bf7-9df6-2a691d587769 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:41.004356 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 200/396] 10.222.0.23 () {56 vars in 1137 bytes} [Thu Oct 13 12:57:40 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:41.004955 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043eb71-5c32-4bf7-9df6-2a691d587769 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:41.019646 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e2d75-d804-4f4c-9336-d0dbc3dc323b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:41.052437 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696120e4-1838-4f7d-bb35-d688bfc16d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:41.053046 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696120e4-1838-4f7d-bb35-d688bfc16d73 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:41.053300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696120e4-1838-4f7d-bb35-d688bfc16d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:41.053572 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696120e4-1838-4f7d-bb35-d688bfc16d73 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:41.054198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696120e4-1838-4f7d-bb35-d688bfc16d73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:41.054497 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696120e4-1838-4f7d-bb35-d688bfc16d73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:41.055897 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696120e4-1838-4f7d-bb35-d688bfc16d73 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:41.067367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 197/397] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:41.068140 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696120e4-1838-4f7d-bb35-d688bfc16d73 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:41.101976 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043eb71-5c32-4bf7-9df6-2a691d587769 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:41.134882 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeaffb4-8cdc-4764-a060-747e01f2a3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:41.135517 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeaffb4-8cdc-4764-a060-747e01f2a3f9 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:41.135795 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeaffb4-8cdc-4764-a060-747e01f2a3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:41.136063 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeaffb4-8cdc-4764-a060-747e01f2a3f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:41.143118 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeaffb4-8cdc-4764-a060-747e01f2a3f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:41.144565 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeaffb4-8cdc-4764-a060-747e01f2a3f9 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:41.156326 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-6eeaffb4-8cdc-4764-a060-747e01f2a3f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:57:41.156326 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:41.156326 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:41.156326 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:41.156326 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:41.156326 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:41.156326 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:41.156326 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:41.156326 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:41.156326 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:41.156326 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:41.156326 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:41.156326 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 13 12:57:41.156326 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 13 12:57:41.156326 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 13 12:57:41.157348 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 13 12:57:41.157348 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:41.157348 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:41.157348 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:41.157348 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:41.157348 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:41.157348 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:41.157348 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:41.157348 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:41.157348 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:41.157348 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:41.157348 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:41.157348 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:41.157348 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:41.157348 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:41.157348 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:41.157348 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:41.162919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 13 12:57:41.162919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 13 12:57:41.162919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 13 12:57:41.162919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 13 12:57:41.162919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 13 12:57:41.162919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 13 12:57:41.162919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:57:41.162919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:41.162919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 201/398] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:57:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:41.162919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeaffb4-8cdc-4764-a060-747e01f2a3f9 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:41.173621 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696120e4-1838-4f7d-bb35-d688bfc16d73 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:41.182180 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:41.198829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:41.208654 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f1e82-3359-430a-8058-e500526253e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:41.209278 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f1e82-3359-430a-8058-e500526253e8 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:41.209989 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f1e82-3359-430a-8058-e500526253e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:41.210271 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f1e82-3359-430a-8058-e500526253e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:41.210918 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f1e82-3359-430a-8058-e500526253e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:41.211246 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f1e82-3359-430a-8058-e500526253e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:41.212618 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f1e82-3359-430a-8058-e500526253e8 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:41.224644 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 198/399] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:41.225229 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f1e82-3359-430a-8058-e500526253e8 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:41.256135 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeaffb4-8cdc-4764-a060-747e01f2a3f9 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:41.290295 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53997f-eac0-4277-ac6c-c4bb0a2ea7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:41.290892 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53997f-eac0-4277-ac6c-c4bb0a2ea7d3 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:41.291198 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53997f-eac0-4277-ac6c-c4bb0a2ea7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:41.291468 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53997f-eac0-4277-ac6c-c4bb0a2ea7d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:41.292105 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53997f-eac0-4277-ac6c-c4bb0a2ea7d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:41.292405 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53997f-eac0-4277-ac6c-c4bb0a2ea7d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:41.294311 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53997f-eac0-4277-ac6c-c4bb0a2ea7d3 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:41.305140 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 202/400] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:41.305873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53997f-eac0-4277-ac6c-c4bb0a2ea7d3 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:41.323263 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f1e82-3359-430a-8058-e500526253e8 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:41.356687 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd3c6df-0e71-484e-af43-cdc7e36ecb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:41.357235 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3c6df-0e71-484e-af43-cdc7e36ecb04 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:41.357514 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3c6df-0e71-484e-af43-cdc7e36ecb04 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:41.357785 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3c6df-0e71-484e-af43-cdc7e36ecb04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:41.367807 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd3c6df-0e71-484e-af43-cdc7e36ecb04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:41.369244 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd3c6df-0e71-484e-af43-cdc7e36ecb04 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:41.382752 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-7cd3c6df-0e71-484e-af43-cdc7e36ecb04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:41.382752 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:41.382752 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:41.382752 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:41.382752 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:41.382752 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:41.382752 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:41.382752 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:41.382752 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:41.382752 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:41.382752 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:41.382752 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:41.382752 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:41.382752 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:41.382752 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:41.382752 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:41.383766 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:41.383766 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:41.383766 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:41.383766 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:41.383766 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:41.383766 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:41.383766 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:41.383766 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:41.383766 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:41.383766 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:41.383766 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:41.383766 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:41.383766 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:41.383766 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:41.383766 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:41.383766 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:41.383766 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:41.389276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:41.389276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:41.389276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:41.389276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 199/401] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:41.389276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd3c6df-0e71-484e-af43-cdc7e36ecb04 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:41.389865 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:41.405314 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53997f-eac0-4277-ac6c-c4bb0a2ea7d3 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:41.426715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:41.437946 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f301a14-5279-44fe-a439-42358026805d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:41.438540 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f301a14-5279-44fe-a439-42358026805d None admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:41.438813 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f301a14-5279-44fe-a439-42358026805d None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:41.439105 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f301a14-5279-44fe-a439-42358026805d None admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:41.439743 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f301a14-5279-44fe-a439-42358026805d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:41.441154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f301a14-5279-44fe-a439-42358026805d None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:41.463324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 203/402] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 13 12:57:41 2022] POST /identity/v3/users => generated 247 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:57:41.464554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f301a14-5279-44fe-a439-42358026805d None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:41.486898 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd3c6df-0e71-484e-af43-cdc7e36ecb04 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:41.530274 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41323c6d-d22e-42d6-8472-ff047a628de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:41.531069 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41323c6d-d22e-42d6-8472-ff047a628de7 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:41.531356 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41323c6d-d22e-42d6-8472-ff047a628de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:41.531631 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41323c6d-d22e-42d6-8472-ff047a628de7 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:41.532605 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41323c6d-d22e-42d6-8472-ff047a628de7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:41.533309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41323c6d-d22e-42d6-8472-ff047a628de7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:41.534856 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41323c6d-d22e-42d6-8472-ff047a628de7 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:41.556485 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 200/403] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 13 12:57:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:41.562598 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41323c6d-d22e-42d6-8472-ff047a628de7 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:41.593529 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f301a14-5279-44fe-a439-42358026805d None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:41.630269 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542bf19-1d9b-47fe-a350-4f714352f044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:41.630823 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542bf19-1d9b-47fe-a350-4f714352f044 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:41.631134 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542bf19-1d9b-47fe-a350-4f714352f044 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:41.631401 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b542bf19-1d9b-47fe-a350-4f714352f044 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:41.638921 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b542bf19-1d9b-47fe-a350-4f714352f044 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:41.640557 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b542bf19-1d9b-47fe-a350-4f714352f044 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:41.643971 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:41.654525 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-b542bf19-1d9b-47fe-a350-4f714352f044 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:57:41.654525 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:41.654525 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:41.654525 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:41.654525 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:41.654525 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:41.654525 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:41.654525 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:41.654525 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:41.654525 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:41.654525 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:41.654525 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:41.654525 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 13 12:57:41.654525 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 13 12:57:41.654525 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 13 12:57:41.655576 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 13 12:57:41.655576 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:41.655576 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:41.655576 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:41.655576 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:41.655576 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:41.655576 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:41.655576 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:41.655576 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:41.655576 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:41.655576 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:41.655576 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:41.655576 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:41.655576 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:41.655576 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:41.655576 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:41.655576 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:41.656646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 13 12:57:41.656646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 13 12:57:41.656646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 13 12:57:41.656646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 13 12:57:41.656646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 13 12:57:41.656646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 13 12:57:41.656646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:57:41.656646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:41.656646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 204/404] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:57:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:41.659920 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542bf19-1d9b-47fe-a350-4f714352f044 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:41.679693 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:41.684115 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41323c6d-d22e-42d6-8472-ff047a628de7 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:41.723637 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756b277-5932-4abc-a5a8-ae50d25c563b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:41.724488 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756b277-5932-4abc-a5a8-ae50d25c563b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:41.724907 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756b277-5932-4abc-a5a8-ae50d25c563b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:41.725315 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756b277-5932-4abc-a5a8-ae50d25c563b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:41.733134 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3756b277-5932-4abc-a5a8-ae50d25c563b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:41.734767 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3756b277-5932-4abc-a5a8-ae50d25c563b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:41.746432 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-3756b277-5932-4abc-a5a8-ae50d25c563b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 13 12:57:41.746432 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:41.746432 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:41.746432 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:41.746432 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:41.746432 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:41.746432 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:41.746432 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:41.746432 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:41.746432 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:41.746432 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:41.746432 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:41.746432 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 13 12:57:41.746432 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 13 12:57:41.746432 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 13 12:57:41.746432 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 13 12:57:41.747407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:41.747407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:41.747407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 13 12:57:41.747407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:57:41.747407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 13 12:57:41.747407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:57:41.747407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:41.747407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:41.747407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:41.747407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:41.747407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:41.747407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:41.747407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:41.747407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:41.747407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:41.747407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:41.747407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:41.747407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:41.748301 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:41.748301 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:41.748301 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 13 12:57:41.748301 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 13 12:57:41.748301 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 13 12:57:41.748301 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 13 12:57:41.748301 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 13 12:57:41.748301 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 13 12:57:41.748301 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 13 12:57:41.748301 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:41.748301 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 201/405] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Oct 13 12:57:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:41.749569 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756b277-5932-4abc-a5a8-ae50d25c563b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:41.771346 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b542bf19-1d9b-47fe-a350-4f714352f044 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:41.805359 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6bac3-4262-48d4-8f22-c04177b340a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:41.805969 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6bac3-4262-48d4-8f22-c04177b340a7 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:41.806254 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6bac3-4262-48d4-8f22-c04177b340a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:41.806522 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6bac3-4262-48d4-8f22-c04177b340a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:41.807193 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad6bac3-4262-48d4-8f22-c04177b340a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:41.807493 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad6bac3-4262-48d4-8f22-c04177b340a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:41.808894 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad6bac3-4262-48d4-8f22-c04177b340a7 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:41.820789 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 205/406] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:41.821594 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6bac3-4262-48d4-8f22-c04177b340a7 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:41.859688 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756b277-5932-4abc-a5a8-ae50d25c563b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:41.884528 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:41.894396 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ce24c-72f8-42b5-afbd-40313f41433c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:41.895150 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ce24c-72f8-42b5-afbd-40313f41433c None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:41.895540 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ce24c-72f8-42b5-afbd-40313f41433c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:41.895872 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ce24c-72f8-42b5-afbd-40313f41433c None admin] PATH_INFO: `/v3/projects/02308892635844b088ecdcb5413a21a0/users/12700d985a724595af630c81e63f86d7/roles/b493ef4ee9d44f449e08251ae75a77d8` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:41.912133 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ce24c-72f8-42b5-afbd-40313f41433c None admin] RBAC: Authorizing `identity:create_grant(project_id=02308892635844b088ecdcb5413a21a0, user_id=12700d985a724595af630c81e63f86d7, role_id=b493ef4ee9d44f449e08251ae75a77d8)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:41.913841 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ce24c-72f8-42b5-afbd-40313f41433c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:41.930989 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 202/407] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 13 12:57:41 2022] PUT /identity/v3/projects/02308892635844b088ecdcb5413a21a0/users/12700d985a724595af630c81e63f86d7/roles/b493ef4ee9d44f449e08251ae75a77d8 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 12:57:41.932123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ce24c-72f8-42b5-afbd-40313f41433c None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:41.932338 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6bac3-4262-48d4-8f22-c04177b340a7 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:41.951560 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:41.981286 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da40635e-b806-481a-af00-c476c0215e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:41.981884 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40635e-b806-481a-af00-c476c0215e82 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:41.982166 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40635e-b806-481a-af00-c476c0215e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:41.982441 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40635e-b806-481a-af00-c476c0215e82 None admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:41.983172 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da40635e-b806-481a-af00-c476c0215e82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:41.983478 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da40635e-b806-481a-af00-c476c0215e82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:41.985056 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da40635e-b806-481a-af00-c476c0215e82 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:42.010259 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 206/408] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 13 12:57:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:42.010259 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da40635e-b806-481a-af00-c476c0215e82 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:42.080261 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ce24c-72f8-42b5-afbd-40313f41433c None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:42.128795 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe27dbd4-7e9e-459c-9553-1f290382fd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:42.129437 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27dbd4-7e9e-459c-9553-1f290382fd71 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:42.129696 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27dbd4-7e9e-459c-9553-1f290382fd71 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:42.129966 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27dbd4-7e9e-459c-9553-1f290382fd71 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:42.130882 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe27dbd4-7e9e-459c-9553-1f290382fd71 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:42.131283 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe27dbd4-7e9e-459c-9553-1f290382fd71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:42.133306 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe27dbd4-7e9e-459c-9553-1f290382fd71 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:42.142609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da40635e-b806-481a-af00-c476c0215e82 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:42.153696 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 203/409] 10.222.0.23 () {56 vars in 1139 bytes} [Thu Oct 13 12:57:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:42.154571 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe27dbd4-7e9e-459c-9553-1f290382fd71 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:42.167215 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:42.190663 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62eface-a370-41fe-8101-e158648307d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:42.191399 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62eface-a370-41fe-8101-e158648307d9 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:42.191684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62eface-a370-41fe-8101-e158648307d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:42.191955 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62eface-a370-41fe-8101-e158648307d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:42.202870 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62eface-a370-41fe-8101-e158648307d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:42.204387 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62eface-a370-41fe-8101-e158648307d9 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:42.221274 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-a62eface-a370-41fe-8101-e158648307d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:57:42.221274 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:42.221274 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:42.221274 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:42.221274 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:42.221274 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:42.221274 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:42.221274 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:42.221274 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:42.221274 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:42.221274 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:42.221274 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:42.221274 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 13 12:57:42.221274 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 13 12:57:42.221274 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 13 12:57:42.222173 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 13 12:57:42.222173 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:42.222173 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:42.222173 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:42.222173 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:42.222173 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:42.222173 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:42.222173 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:42.222173 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:42.222173 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:42.222173 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:42.222173 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:42.222173 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:42.222173 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:42.222173 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:42.222173 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:42.222173 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:42.223004 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 13 12:57:42.223004 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 13 12:57:42.223004 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 13 12:57:42.223004 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 13 12:57:42.223004 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 13 12:57:42.223004 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 13 12:57:42.223004 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:57:42.223004 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:42.227139 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 207/410] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:57:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:42.229232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62eface-a370-41fe-8101-e158648307d9 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:42.242025 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:42.280787 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe27dbd4-7e9e-459c-9553-1f290382fd71 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:42.316880 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3dd894-8140-482c-bfd8-dc926b11cea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:42.317897 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3dd894-8140-482c-bfd8-dc926b11cea9 None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:42.318445 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3dd894-8140-482c-bfd8-dc926b11cea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:42.318964 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3dd894-8140-482c-bfd8-dc926b11cea9 None admin] PATH_INFO: `/v3/projects/02308892635844b088ecdcb5413a21a0/users/45ce234c5b4643b2ac6341e0928f3830/roles/b493ef4ee9d44f449e08251ae75a77d8` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:42.334893 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3dd894-8140-482c-bfd8-dc926b11cea9 None admin] RBAC: Authorizing `identity:create_grant(project_id=02308892635844b088ecdcb5413a21a0, user_id=45ce234c5b4643b2ac6341e0928f3830, role_id=b493ef4ee9d44f449e08251ae75a77d8)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:42.336659 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3dd894-8140-482c-bfd8-dc926b11cea9 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:42.338559 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62eface-a370-41fe-8101-e158648307d9 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:42.353006 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 204/411] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 13 12:57:42 2022] PUT /identity/v3/projects/02308892635844b088ecdcb5413a21a0/users/45ce234c5b4643b2ac6341e0928f3830/roles/b493ef4ee9d44f449e08251ae75a77d8 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 12:57:42.354276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3dd894-8140-482c-bfd8-dc926b11cea9 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:42.401119 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c592bea-94d2-4457-b995-9af00ee3c517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:42.402819 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c592bea-94d2-4457-b995-9af00ee3c517 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:42.402819 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c592bea-94d2-4457-b995-9af00ee3c517 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:42.403008 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c592bea-94d2-4457-b995-9af00ee3c517 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:42.424307 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c592bea-94d2-4457-b995-9af00ee3c517 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:42.425994 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c592bea-94d2-4457-b995-9af00ee3c517 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:42.454669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-4c592bea-94d2-4457-b995-9af00ee3c517 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:42.454669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:42.454669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:42.454669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:42.454669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:42.454669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:42.454669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:42.454669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:42.454669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:42.454669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:42.454669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:42.454669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:42.454669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:42.454669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:42.454669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:42.454669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:42.455695 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:42.455695 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:42.455695 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:42.455695 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:42.455695 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:42.455695 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:42.455695 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:42.455695 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:42.455695 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:42.455695 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:42.455695 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:42.455695 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:42.455695 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:42.455695 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:42.455695 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:42.455695 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:42.455695 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:42.457002 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:42.457002 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:42.457002 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:42.457002 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 208/412] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:42.462750 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c592bea-94d2-4457-b995-9af00ee3c517 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:42.476880 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3dd894-8140-482c-bfd8-dc926b11cea9 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:42.496757 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:42.520114 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53762cde-88c9-4f65-a4ed-9190bce2ef01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:42.521104 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53762cde-88c9-4f65-a4ed-9190bce2ef01 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:42.521581 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53762cde-88c9-4f65-a4ed-9190bce2ef01 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:42.522048 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53762cde-88c9-4f65-a4ed-9190bce2ef01 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:42.522991 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53762cde-88c9-4f65-a4ed-9190bce2ef01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:42.523597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53762cde-88c9-4f65-a4ed-9190bce2ef01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:42.525222 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53762cde-88c9-4f65-a4ed-9190bce2ef01 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:42.525761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:42.545307 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 205/413] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 13 12:57:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:42.546988 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53762cde-88c9-4f65-a4ed-9190bce2ef01 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:42.659830 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53762cde-88c9-4f65-a4ed-9190bce2ef01 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:42.681737 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c592bea-94d2-4457-b995-9af00ee3c517 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:42.696868 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261c2a3-918b-4454-b993-df9aad0bfad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:42.697889 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261c2a3-918b-4454-b993-df9aad0bfad5 None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:42.698393 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261c2a3-918b-4454-b993-df9aad0bfad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:42.698885 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261c2a3-918b-4454-b993-df9aad0bfad5 None admin] PATH_INFO: `/v3/projects/02308892635844b088ecdcb5413a21a0/users/2398a599165a46adb5718d98c897f238/roles/b493ef4ee9d44f449e08251ae75a77d8` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:42.715108 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7261c2a3-918b-4454-b993-df9aad0bfad5 None admin] RBAC: Authorizing `identity:create_grant(project_id=02308892635844b088ecdcb5413a21a0, user_id=2398a599165a46adb5718d98c897f238, role_id=b493ef4ee9d44f449e08251ae75a77d8)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:42.717181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7261c2a3-918b-4454-b993-df9aad0bfad5 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:42.718710 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bcf241-b949-4a99-942b-5ea34ba7f97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:42.719358 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bcf241-b949-4a99-942b-5ea34ba7f97b None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:42.719635 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bcf241-b949-4a99-942b-5ea34ba7f97b None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:42.719904 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bcf241-b949-4a99-942b-5ea34ba7f97b None admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:42.720548 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bcf241-b949-4a99-942b-5ea34ba7f97b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:42.720852 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bcf241-b949-4a99-942b-5ea34ba7f97b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:42.722498 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bcf241-b949-4a99-942b-5ea34ba7f97b None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:42.735797 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 209/414] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:42.740192 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 206/415] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 13 12:57:42 2022] PUT /identity/v3/projects/02308892635844b088ecdcb5413a21a0/users/2398a599165a46adb5718d98c897f238/roles/b493ef4ee9d44f449e08251ae75a77d8 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 12:57:42.741397 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bcf241-b949-4a99-942b-5ea34ba7f97b None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:42.844849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:42.854349 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bcf241-b949-4a99-942b-5ea34ba7f97b None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:42.890025 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54baf18b-815b-401a-b0ed-f511d38fe505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:42.890657 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54baf18b-815b-401a-b0ed-f511d38fe505 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:42.890937 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54baf18b-815b-401a-b0ed-f511d38fe505 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:42.891336 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54baf18b-815b-401a-b0ed-f511d38fe505 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:42.898503 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54baf18b-815b-401a-b0ed-f511d38fe505 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:42.900013 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54baf18b-815b-401a-b0ed-f511d38fe505 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:42.913225 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-54baf18b-815b-401a-b0ed-f511d38fe505 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:57:42.913225 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:42.913225 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:42.913225 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:42.913225 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:42.913225 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:42.913225 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:42.913225 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:42.913225 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:42.913225 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:42.913225 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:42.913225 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:42.913225 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 13 12:57:42.913225 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 13 12:57:42.913225 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 13 12:57:42.914067 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 13 12:57:42.914067 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:42.914067 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:42.914067 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:42.914067 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:42.914067 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:42.914067 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:42.914067 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:42.914067 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:42.914067 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:42.914067 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:42.914067 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:42.914067 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:42.914067 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:42.914067 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:42.914067 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:42.914067 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:42.915346 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 13 12:57:42.915346 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 13 12:57:42.915346 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 13 12:57:42.915346 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 13 12:57:42.915346 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 13 12:57:42.915346 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 13 12:57:42.915346 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:57:42.915346 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:42.915346 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 210/416] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:57:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:42.922214 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261c2a3-918b-4454-b993-df9aad0bfad5 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:42.973594 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:43.054477 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261c2a3-918b-4454-b993-df9aad0bfad5 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:43.093060 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ac8b7-6c9f-4c57-8315-8865f0a55b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:43.094456 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ac8b7-6c9f-4c57-8315-8865f0a55b4f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:43.094935 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ac8b7-6c9f-4c57-8315-8865f0a55b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:43.095426 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ac8b7-6c9f-4c57-8315-8865f0a55b4f None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:43.096333 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ac8b7-6c9f-4c57-8315-8865f0a55b4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:43.096834 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ac8b7-6c9f-4c57-8315-8865f0a55b4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:43.098458 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ac8b7-6c9f-4c57-8315-8865f0a55b4f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:43.111528 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 207/417] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 13 12:57:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:43.117374 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54baf18b-815b-401a-b0ed-f511d38fe505 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:43.240222 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54baf18b-815b-401a-b0ed-f511d38fe505 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:43.278828 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638c2b5-5b71-4731-9cd7-12085609d5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:43.279529 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638c2b5-5b71-4731-9cd7-12085609d5fb None admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:43.279810 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638c2b5-5b71-4731-9cd7-12085609d5fb None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:43.280100 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638c2b5-5b71-4731-9cd7-12085609d5fb None admin] PATH_INFO: `/v3/projects/02308892635844b088ecdcb5413a21a0/users/846251a661c04d3f9f195c4ac09b4189/roles/b493ef4ee9d44f449e08251ae75a77d8` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:43.300440 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638c2b5-5b71-4731-9cd7-12085609d5fb None admin] RBAC: Authorizing `identity:create_grant(project_id=02308892635844b088ecdcb5413a21a0, user_id=846251a661c04d3f9f195c4ac09b4189, role_id=b493ef4ee9d44f449e08251ae75a77d8)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:43.302296 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638c2b5-5b71-4731-9cd7-12085609d5fb None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:43.322244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 211/418] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 13 12:57:43 2022] PUT /identity/v3/projects/02308892635844b088ecdcb5413a21a0/users/846251a661c04d3f9f195c4ac09b4189/roles/b493ef4ee9d44f449e08251ae75a77d8 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 12:57:43.849185 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5613418-9901-4c86-ac85-5800c377ec63 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:43.849872 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5613418-9901-4c86-ac85-5800c377ec63 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:43.850406 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5613418-9901-4c86-ac85-5800c377ec63 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:43.852020 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 208/419] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:43.858258 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca39b1e-0503-463a-a16a-49ce560342cc None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:43.858772 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca39b1e-0503-463a-a16a-49ce560342cc None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:43.859268 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca39b1e-0503-463a-a16a-49ce560342cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:43.865836 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:43.884036 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-1ca39b1e-0503-463a-a16a-49ce560342cc None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:43.885252 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-1ca39b1e-0503-463a-a16a-49ce560342cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:43.992739 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 212/420] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:44.338774 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5613418-9901-4c86-ac85-5800c377ec63 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:44.425300 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb1788-4923-47b1-ba8e-fd491f62ac55 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:44.425840 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb1788-4923-47b1-ba8e-fd491f62ac55 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:44.425840 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb1788-4923-47b1-ba8e-fd491f62ac55 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:44.427208 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 213/421] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:44.432470 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0257b-5c2f-4586-9b78-abcb05ec10e1 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:44.432765 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0257b-5c2f-4586-9b78-abcb05ec10e1 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:44.433018 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0257b-5c2f-4586-9b78-abcb05ec10e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:44.455309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:44.475149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5613418-9901-4c86-ac85-5800c377ec63 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:44.484040 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-c5f0257b-5c2f-4586-9b78-abcb05ec10e1 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:44.485226 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-c5f0257b-5c2f-4586-9b78-abcb05ec10e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:44.523796 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fcd760-309a-44b5-b0ed-4f1191bc66eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:44.524444 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fcd760-309a-44b5-b0ed-4f1191bc66eb None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:44.524772 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fcd760-309a-44b5-b0ed-4f1191bc66eb None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:44.525044 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fcd760-309a-44b5-b0ed-4f1191bc66eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:44.544287 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fcd760-309a-44b5-b0ed-4f1191bc66eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:44.557098 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-d9fcd760-309a-44b5-b0ed-4f1191bc66eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 13 12:57:44.557098 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:44.557098 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:44.557098 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:44.557098 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:44.557098 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:44.557098 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:44.557098 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:44.557098 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:44.557098 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:44.557098 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:44.557098 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:44.557098 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:44.557098 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:44.557098 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:44.557098 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:44.558051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:44.558051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:44.558051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:44.558051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:44.558051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:44.558051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:44.558051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:44.558051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:44.558051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:44.558051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:44.558051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:44.558051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:44.558051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:44.558051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:44.558051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:44.558051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:44.559975 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:44.559975 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:44.559975 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:44.559975 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:44.559975 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:44.559975 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:44.559975 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:44.559975 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:44.559975 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:44.559975 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:44.559975 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 13 12:57:44.559975 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:44.559975 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 209/422] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Oct 13 12:57:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:44.567205 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fcd760-309a-44b5-b0ed-4f1191bc66eb None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:44.608860 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 214/423] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 13 12:57:44.691044 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fcd760-309a-44b5-b0ed-4f1191bc66eb None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:44.733115 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9003e6-cdce-4387-85dd-7cf170bbf7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:44.733926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9003e6-cdce-4387-85dd-7cf170bbf7fa None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:44.734347 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9003e6-cdce-4387-85dd-7cf170bbf7fa None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:44.734786 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9003e6-cdce-4387-85dd-7cf170bbf7fa None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:44.735627 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9003e6-cdce-4387-85dd-7cf170bbf7fa None admin] RBAC: Adding query filter params (name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:44.736072 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9003e6-cdce-4387-85dd-7cf170bbf7fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:44.737634 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9003e6-cdce-4387-85dd-7cf170bbf7fa None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:44.750824 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 210/424] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Oct 13 12:57:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:57:44.752424 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0257b-5c2f-4586-9b78-abcb05ec10e1 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:44.850324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:44.905926 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0257b-5c2f-4586-9b78-abcb05ec10e1 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:44.953346 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc4a31-9c4a-4575-af11-f977289b3ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:44.953982 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc4a31-9c4a-4575-af11-f977289b3ded None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:44.954259 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc4a31-9c4a-4575-af11-f977289b3ded None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:44.954530 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc4a31-9c4a-4575-af11-f977289b3ded None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:44.969089 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc4a31-9c4a-4575-af11-f977289b3ded None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:44.970555 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc4a31-9c4a-4575-af11-f977289b3ded None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:45.000873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-f6cc4a31-9c4a-4575-af11-f977289b3ded None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:45.000873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:45.000873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:45.000873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:45.000873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:45.000873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:45.000873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:45.000873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:45.000873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:45.000873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:45.000873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:45.000873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:45.000873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:45.000873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:45.000873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:45.000873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:45.002027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:45.002027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:45.002027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:45.002027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:45.002027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:45.002027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:45.002027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:45.002027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:45.002027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:45.002027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:45.002027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:45.002027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:45.002027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:45.002027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:45.002027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:45.002027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:45.002027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:45.003208 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:45.003208 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:45.003208 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:45.003208 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 215/425] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:45.009070 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9003e6-cdce-4387-85dd-7cf170bbf7fa None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:45.010793 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc4a31-9c4a-4575-af11-f977289b3ded None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:45.021991 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:45.118648 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9003e6-cdce-4387-85dd-7cf170bbf7fa None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:45.160751 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ac174-0af2-44dc-9855-a185d7c2badf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:45.161303 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ac174-0af2-44dc-9855-a185d7c2badf None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:45.161581 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ac174-0af2-44dc-9855-a185d7c2badf None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:45.161853 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ac174-0af2-44dc-9855-a185d7c2badf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:45.182408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ac174-0af2-44dc-9855-a185d7c2badf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:45.188372 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-771ac174-0af2-44dc-9855-a185d7c2badf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 13 12:57:45.188372 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:45.188372 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:45.188372 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:45.188372 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:45.188372 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:45.188372 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:45.188372 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:45.188372 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:45.188372 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:45.188372 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:45.188372 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:45.188372 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:45.188372 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:45.188372 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:45.188372 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:45.189475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:45.189475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:45.189475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:45.189475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:45.189475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:45.189475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:45.189475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:45.189475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:45.189475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:45.189475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:45.189475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:45.189475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:45.189475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:45.189475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:45.189475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:45.189475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:45.190506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:45.190506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:45.190506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:45.190506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:45.190506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:45.190506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:45.190506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:45.190506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:45.190506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:45.190506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:45.190506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 13 12:57:45.190506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:45.190506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 211/426] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Oct 13 12:57:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:45.198001 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ac174-0af2-44dc-9855-a185d7c2badf None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:45.224660 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc4a31-9c4a-4575-af11-f977289b3ded None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:45.251917 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:45.290841 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31362d-9dca-4a43-bdf9-35fc091b9097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:45.291486 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31362d-9dca-4a43-bdf9-35fc091b9097 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:45.291767 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31362d-9dca-4a43-bdf9-35fc091b9097 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:45.292036 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31362d-9dca-4a43-bdf9-35fc091b9097 None admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:45.293359 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e31362d-9dca-4a43-bdf9-35fc091b9097 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:45.293654 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e31362d-9dca-4a43-bdf9-35fc091b9097 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:45.295048 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e31362d-9dca-4a43-bdf9-35fc091b9097 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:45.307614 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 216/427] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:57:45.320118 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31362d-9dca-4a43-bdf9-35fc091b9097 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:45.325138 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:45.362734 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ac174-0af2-44dc-9855-a185d7c2badf None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:45.399840 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae8caa2-456e-402a-81b9-fa0234d11c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:45.400705 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8caa2-456e-402a-81b9-fa0234d11c35 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:45.401208 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8caa2-456e-402a-81b9-fa0234d11c35 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:45.401619 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8caa2-456e-402a-81b9-fa0234d11c35 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:45.402453 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae8caa2-456e-402a-81b9-fa0234d11c35 None admin] RBAC: Adding query filter params (name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:45.402895 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae8caa2-456e-402a-81b9-fa0234d11c35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:45.404424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae8caa2-456e-402a-81b9-fa0234d11c35 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:45.414789 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 212/428] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Oct 13 12:57:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:45.423174 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae8caa2-456e-402a-81b9-fa0234d11c35 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:45.491428 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31362d-9dca-4a43-bdf9-35fc091b9097 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:45.537628 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1907d5-23aa-4687-b5bb-9fea335f782f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:45.538262 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1907d5-23aa-4687-b5bb-9fea335f782f None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:45.538542 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1907d5-23aa-4687-b5bb-9fea335f782f None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:45.538815 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1907d5-23aa-4687-b5bb-9fea335f782f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:45.539244 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae8caa2-456e-402a-81b9-fa0234d11c35 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:45.549022 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1907d5-23aa-4687-b5bb-9fea335f782f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:45.550473 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1907d5-23aa-4687-b5bb-9fea335f782f None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:45.568383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-3b1907d5-23aa-4687-b5bb-9fea335f782f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 13 12:57:45.568383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:45.568383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:45.568383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:45.568383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:45.568383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:45.568383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:45.568383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:45.568383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:45.568383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:45.568383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:45.568383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:45.568383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 13 12:57:45.568383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 13 12:57:45.568383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 13 12:57:45.568383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 13 12:57:45.569318 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:45.569318 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:45.569318 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 13 12:57:45.569318 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:57:45.569318 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 13 12:57:45.569318 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:57:45.569318 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:45.569318 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:45.569318 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:45.569318 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:45.569318 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:45.569318 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:45.569318 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:45.569318 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:45.569318 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:45.569318 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:45.569318 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:45.569318 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:45.574192 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:45.574192 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:45.574192 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 13 12:57:45.574192 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 13 12:57:45.574192 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 13 12:57:45.574192 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 13 12:57:45.574192 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 13 12:57:45.574192 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 13 12:57:45.574192 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 13 12:57:45.574192 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:45.574192 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 217/429] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 13 12:57:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:45.574192 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a3fa0-f919-40d6-a5ea-2fdd5f3fe9d3 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:45.574192 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a3fa0-f919-40d6-a5ea-2fdd5f3fe9d3 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:45.576734 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a3fa0-f919-40d6-a5ea-2fdd5f3fe9d3 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:45.576734 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 218/430] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:45.577345 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a3fa0-f919-40d6-a5ea-2fdd5f3fe9d3 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:45.583333 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:45.592014 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917caa0e-0afa-4208-b3bf-98a1eb1dfb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:45.592654 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917caa0e-0afa-4208-b3bf-98a1eb1dfb08 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:45.592936 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917caa0e-0afa-4208-b3bf-98a1eb1dfb08 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:45.593216 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917caa0e-0afa-4208-b3bf-98a1eb1dfb08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:45.606402 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917caa0e-0afa-4208-b3bf-98a1eb1dfb08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:45.607896 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917caa0e-0afa-4208-b3bf-98a1eb1dfb08 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:45.619997 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:45.622679 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-917caa0e-0afa-4208-b3bf-98a1eb1dfb08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:45.622679 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:45.622679 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:45.622679 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:45.622679 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:45.622679 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:45.622679 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:45.622679 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:45.622679 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:45.622679 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:45.622679 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:45.622679 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:45.622679 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:45.622679 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:45.622679 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:45.622679 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:45.623611 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:45.623611 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:45.623611 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:45.623611 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:45.623611 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:45.623611 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:45.623611 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:45.623611 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:45.623611 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:45.623611 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:45.623611 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:45.623611 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:45.623611 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:45.623611 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:45.623611 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:45.623611 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:45.623611 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:45.627775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:45.627775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:45.627775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:45.627775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 213/431] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:57:45.627775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f6a4b-a50f-4d75-a4a3-20f7e7ebc09c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:45.627775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f6a4b-a50f-4d75-a4a3-20f7e7ebc09c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:45.627775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f6a4b-a50f-4d75-a4a3-20f7e7ebc09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:45.644913 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-219f6a4b-a50f-4d75-a4a3-20f7e7ebc09c None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:45.646104 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-219f6a4b-a50f-4d75-a4a3-20f7e7ebc09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:45.698670 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a3fa0-f919-40d6-a5ea-2fdd5f3fe9d3 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:45.717578 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 214/432] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:45.719236 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a69ffc-b0e9-406e-bee1-fce6012a82d2 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:45.719236 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a69ffc-b0e9-406e-bee1-fce6012a82d2 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:45.719489 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a69ffc-b0e9-406e-bee1-fce6012a82d2 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:45.720719 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 215/433] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:45.721413 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a69ffc-b0e9-406e-bee1-fce6012a82d2 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:45.743744 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd89c8b0-07eb-4b3e-b715-cf51f67bd8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:45.744367 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd89c8b0-07eb-4b3e-b715-cf51f67bd8a4 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:45.744740 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd89c8b0-07eb-4b3e-b715-cf51f67bd8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:45.745262 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd89c8b0-07eb-4b3e-b715-cf51f67bd8a4 None admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:45.745951 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd89c8b0-07eb-4b3e-b715-cf51f67bd8a4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:45.746316 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd89c8b0-07eb-4b3e-b715-cf51f67bd8a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:45.747885 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd89c8b0-07eb-4b3e-b715-cf51f67bd8a4 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:45.774389 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 219/434] 10.222.0.23 () {56 vars in 1135 bytes} [Thu Oct 13 12:57:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:45.777732 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cdfc6-1048-43fd-b16c-dd75bd8475f0 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:45.778018 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cdfc6-1048-43fd-b16c-dd75bd8475f0 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:45.778346 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238cdfc6-1048-43fd-b16c-dd75bd8475f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:45.798369 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-238cdfc6-1048-43fd-b16c-dd75bd8475f0 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:45.799640 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-238cdfc6-1048-43fd-b16c-dd75bd8475f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:45.836067 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a69ffc-b0e9-406e-bee1-fce6012a82d2 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:45.850517 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:45.873688 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac19c864-1479-448e-a908-1142a9572aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:45.874641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19c864-1479-448e-a908-1142a9572aa2 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:45.875163 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19c864-1479-448e-a908-1142a9572aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:45.875630 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19c864-1479-448e-a908-1142a9572aa2 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:45.876502 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac19c864-1479-448e-a908-1142a9572aa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:45.877032 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac19c864-1479-448e-a908-1142a9572aa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:45.878635 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac19c864-1479-448e-a908-1142a9572aa2 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:45.883647 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 220/435] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:45.884732 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238cdfc6-1048-43fd-b16c-dd75bd8475f0 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:45.892843 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 216/436] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:45.898047 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac19c864-1479-448e-a908-1142a9572aa2 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:45.932509 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:46.041145 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238cdfc6-1048-43fd-b16c-dd75bd8475f0 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:46.048793 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac19c864-1479-448e-a908-1142a9572aa2 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:46.075718 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63140005-c2f6-4c57-98fa-d40c3a8c37c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:46.076344 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63140005-c2f6-4c57-98fa-d40c3a8c37c5 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:46.076627 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63140005-c2f6-4c57-98fa-d40c3a8c37c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:46.076901 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63140005-c2f6-4c57-98fa-d40c3a8c37c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:46.085726 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f77c53-c46d-4aa3-8925-c9f844903b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:46.086823 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f77c53-c46d-4aa3-8925-c9f844903b43 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:46.087287 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63140005-c2f6-4c57-98fa-d40c3a8c37c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:46.087588 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f77c53-c46d-4aa3-8925-c9f844903b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:46.088025 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f77c53-c46d-4aa3-8925-c9f844903b43 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:46.088738 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63140005-c2f6-4c57-98fa-d40c3a8c37c5 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:46.096425 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f77c53-c46d-4aa3-8925-c9f844903b43 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:46.097844 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f77c53-c46d-4aa3-8925-c9f844903b43 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:46.105334 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-63140005-c2f6-4c57-98fa-d40c3a8c37c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:46.105334 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:46.105334 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:46.105334 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:46.105334 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:46.105334 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:46.105334 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:46.105334 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:46.105334 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:46.105334 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:46.105334 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:46.105334 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:46.105334 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:46.105334 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:46.105334 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:46.105334 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:46.106212 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:46.106212 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:46.106212 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:46.106212 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:46.106212 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:46.106212 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:46.106212 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:46.106212 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:46.106212 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:46.106212 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:46.106212 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:46.106212 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:46.106212 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:46.106212 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:46.106212 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:46.106212 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:46.106212 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:46.107042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:46.107042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:46.107042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:46.107042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 221/437] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:46.107292 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63140005-c2f6-4c57-98fa-d40c3a8c37c5 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:46.110241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-b0f77c53-c46d-4aa3-8925-c9f844903b43 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 13 12:57:46.110241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:46.110241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:46.110241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:46.110241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:46.110241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:46.110241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:46.110241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:46.110241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:46.110241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:46.110241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:46.110241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:46.110241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 13 12:57:46.110241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 13 12:57:46.110241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 13 12:57:46.110241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 13 12:57:46.111149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:46.111149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:46.111149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 13 12:57:46.111149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:57:46.111149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 13 12:57:46.111149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:57:46.111149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:46.111149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:46.111149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:46.111149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:46.111149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:46.111149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:46.111149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:46.111149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:46.111149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:46.111149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:46.111149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:46.111149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:46.112009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:46.112009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:46.112009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 13 12:57:46.112009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 13 12:57:46.112009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 13 12:57:46.112009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 13 12:57:46.112009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 13 12:57:46.112009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 13 12:57:46.112009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 13 12:57:46.112009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:46.115417 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 217/438] 10.222.0.23 () {56 vars in 1093 bytes} [Thu Oct 13 12:57:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:57:46.116727 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd3641-0aa2-43c3-93b5-4816f79f86a4 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:46.117012 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd3641-0aa2-43c3-93b5-4816f79f86a4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:46.117288 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd3641-0aa2-43c3-93b5-4816f79f86a4 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:46.118540 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 218/439] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:46.119256 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd3641-0aa2-43c3-93b5-4816f79f86a4 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:46.158377 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:46.193316 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:46.218535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63140005-c2f6-4c57-98fa-d40c3a8c37c5 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:46.232802 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd3641-0aa2-43c3-93b5-4816f79f86a4 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:46.252491 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835aa81-a8af-44d0-b608-edf288ca39fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:46.253117 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835aa81-a8af-44d0-b608-edf288ca39fe None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:46.253407 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835aa81-a8af-44d0-b608-edf288ca39fe None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:46.253678 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835aa81-a8af-44d0-b608-edf288ca39fe None admin] PATH_INFO: `/v3/roles/service` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:46.263505 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835aa81-a8af-44d0-b608-edf288ca39fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:46.266371 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d993372-c0ae-4d19-bf6f-1d806460e129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:46.266967 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d993372-c0ae-4d19-bf6f-1d806460e129 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:46.267272 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d993372-c0ae-4d19-bf6f-1d806460e129 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:46.267546 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d993372-c0ae-4d19-bf6f-1d806460e129 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:46.268181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d993372-c0ae-4d19-bf6f-1d806460e129 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:46.268482 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d993372-c0ae-4d19-bf6f-1d806460e129 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:46.268644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-4835aa81-a8af-44d0-b608-edf288ca39fe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 13 12:57:46.268644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:46.268644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:46.268644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:46.268644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:46.268644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:46.268644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:46.268644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:46.268644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:46.268644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:46.268644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:46.268644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:46.268644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:46.268644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:46.268644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:46.268644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:46.269635 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:46.269635 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:46.269635 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:46.269635 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:46.269635 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:46.269635 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:46.269635 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:46.269635 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:46.269635 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:46.269635 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:46.269635 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:46.269635 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:46.269635 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:46.269635 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:46.269635 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:46.269635 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:46.270430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:46.270430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:46.270430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:46.270430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:46.270430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:46.270430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:46.270430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:46.270430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:46.270430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:46.270430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:46.270430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 13 12:57:46.270430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:46.270430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 222/440] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Oct 13 12:57:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:46.271096 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835aa81-a8af-44d0-b608-edf288ca39fe None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:46.274634 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d993372-c0ae-4d19-bf6f-1d806460e129 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:46.285902 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 219/441] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:46.287091 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3ef68-a0c1-40f4-a9a9-6790b0d0daab None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:46.287382 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3ef68-a0c1-40f4-a9a9-6790b0d0daab None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:46.287651 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3ef68-a0c1-40f4-a9a9-6790b0d0daab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:46.305197 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-51a3ef68-a0c1-40f4-a9a9-6790b0d0daab None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:46.306378 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-51a3ef68-a0c1-40f4-a9a9-6790b0d0daab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:46.368878 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835aa81-a8af-44d0-b608-edf288ca39fe None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:46.369835 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 220/442] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:46.370662 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3ef68-a0c1-40f4-a9a9-6790b0d0daab None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:46.402758 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc556d1-3da7-43ea-be96-ba8b9c1dd229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:46.403386 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc556d1-3da7-43ea-be96-ba8b9c1dd229 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:46.403666 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc556d1-3da7-43ea-be96-ba8b9c1dd229 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:46.403933 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc556d1-3da7-43ea-be96-ba8b9c1dd229 None admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:46.404657 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc556d1-3da7-43ea-be96-ba8b9c1dd229 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:46.404961 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc556d1-3da7-43ea-be96-ba8b9c1dd229 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:46.405565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:46.406407 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc556d1-3da7-43ea-be96-ba8b9c1dd229 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:46.408027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:46.424137 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 223/443] 10.222.0.23 () {56 vars in 1133 bytes} [Thu Oct 13 12:57:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:46.424942 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc556d1-3da7-43ea-be96-ba8b9c1dd229 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:46.493149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3ef68-a0c1-40f4-a9a9-6790b0d0daab None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:46.529673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc556d1-3da7-43ea-be96-ba8b9c1dd229 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:46.531617 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528b3f6-f4c0-4581-b5db-c6c7a4b8588a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:46.532252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528b3f6-f4c0-4581-b5db-c6c7a4b8588a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:46.532533 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528b3f6-f4c0-4581-b5db-c6c7a4b8588a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:46.532809 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528b3f6-f4c0-4581-b5db-c6c7a4b8588a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:46.544182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2528b3f6-f4c0-4581-b5db-c6c7a4b8588a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:46.549591 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-2528b3f6-f4c0-4581-b5db-c6c7a4b8588a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 13 12:57:46.549591 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:46.549591 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:46.549591 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:46.549591 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:46.549591 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:46.549591 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:46.549591 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:46.549591 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:46.549591 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:46.549591 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:46.549591 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:46.549591 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:46.549591 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:46.549591 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:46.549591 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:46.550464 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:46.550464 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:46.550464 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:46.550464 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:46.550464 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:46.550464 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:46.550464 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:46.550464 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:46.550464 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:46.550464 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:46.550464 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:46.550464 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:46.550464 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:46.550464 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:46.550464 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:46.550464 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:46.551293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:46.551293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:46.551293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:46.551293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:46.551293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:46.551293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:46.551293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:46.551293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:46.551293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:46.551293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:46.551293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 13 12:57:46.551293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:46.551293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 221/444] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Oct 13 12:57:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:46.551948 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528b3f6-f4c0-4581-b5db-c6c7a4b8588a None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:46.569459 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56622732-e5c7-4964-870a-1262859359f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:46.570008 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56622732-e5c7-4964-870a-1262859359f2 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:46.570274 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56622732-e5c7-4964-870a-1262859359f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:46.570550 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56622732-e5c7-4964-870a-1262859359f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:46.571245 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56622732-e5c7-4964-870a-1262859359f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:46.571548 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56622732-e5c7-4964-870a-1262859359f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:46.572912 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56622732-e5c7-4964-870a-1262859359f2 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:46.582011 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 224/445] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Oct 13 12:57:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:46.583043 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56622732-e5c7-4964-870a-1262859359f2 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:46.637821 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:46.645135 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:46.663972 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528b3f6-f4c0-4581-b5db-c6c7a4b8588a None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:46.682696 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56622732-e5c7-4964-870a-1262859359f2 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:46.696468 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3718991b-cfc7-4351-a908-ca9dc85eb220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:46.697088 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718991b-cfc7-4351-a908-ca9dc85eb220 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:46.697368 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718991b-cfc7-4351-a908-ca9dc85eb220 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:46.697640 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718991b-cfc7-4351-a908-ca9dc85eb220 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:46.704552 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3718991b-cfc7-4351-a908-ca9dc85eb220 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:46.705988 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3718991b-cfc7-4351-a908-ca9dc85eb220 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:46.715328 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e962e32a-6e5a-45e1-a3d4-745ef9bba49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:46.715923 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962e32a-6e5a-45e1-a3d4-745ef9bba49e None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:46.716200 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962e32a-6e5a-45e1-a3d4-745ef9bba49e None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:46.716470 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962e32a-6e5a-45e1-a3d4-745ef9bba49e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:46.717525 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-3718991b-cfc7-4351-a908-ca9dc85eb220 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:57:46.717525 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:46.717525 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:46.717525 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:46.717525 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:46.717525 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:46.717525 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:46.717525 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:46.717525 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:46.717525 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:46.717525 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:46.717525 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:46.717525 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 13 12:57:46.717525 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 13 12:57:46.717525 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 13 12:57:46.718480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 13 12:57:46.718480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:46.718480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:46.718480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:46.718480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:46.718480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:46.718480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:46.718480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:46.718480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:46.718480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:46.718480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:46.718480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:46.718480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:46.718480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:46.718480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:46.718480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:46.718480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:46.719371 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 13 12:57:46.719371 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 13 12:57:46.719371 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 13 12:57:46.719371 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 13 12:57:46.719371 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 13 12:57:46.719371 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 13 12:57:46.719371 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:57:46.719371 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:46.719371 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 222/446] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:57:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:46.719371 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3718991b-cfc7-4351-a908-ca9dc85eb220 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:46.787927 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e962e32a-6e5a-45e1-a3d4-745ef9bba49e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:46.789391 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e962e32a-6e5a-45e1-a3d4-745ef9bba49e None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:46.817601 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3718991b-cfc7-4351-a908-ca9dc85eb220 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:46.842537 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:46.850433 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e18371-b728-4527-a401-8c6846b8bf84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:46.851030 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e18371-b728-4527-a401-8c6846b8bf84 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:46.851339 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e18371-b728-4527-a401-8c6846b8bf84 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:46.851608 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e18371-b728-4527-a401-8c6846b8bf84 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:46.852255 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e18371-b728-4527-a401-8c6846b8bf84 None admin] RBAC: Adding query filter params (name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:46.852555 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e18371-b728-4527-a401-8c6846b8bf84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:46.853904 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e18371-b728-4527-a401-8c6846b8bf84 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:46.861509 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 223/447] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Oct 13 12:57:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:46.862275 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e18371-b728-4527-a401-8c6846b8bf84 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:46.867101 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:46.888063 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 225/448] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 13 12:57:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 13 12:57:46.889011 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e962e32a-6e5a-45e1-a3d4-745ef9bba49e None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:46.968145 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e18371-b728-4527-a401-8c6846b8bf84 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:46.984954 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e962e32a-6e5a-45e1-a3d4-745ef9bba49e None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:47.000525 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000bc22-5e52-4955-a917-eaad09292cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:47.001139 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000bc22-5e52-4955-a917-eaad09292cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:47.001415 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000bc22-5e52-4955-a917-eaad09292cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:47.001686 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000bc22-5e52-4955-a917-eaad09292cf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:47.012638 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1000bc22-5e52-4955-a917-eaad09292cf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:47.014062 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1000bc22-5e52-4955-a917-eaad09292cf0 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:47.017920 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088c274-7ccf-447b-8027-e200e99caad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:47.018543 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088c274-7ccf-447b-8027-e200e99caad2 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:47.018824 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088c274-7ccf-447b-8027-e200e99caad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:47.019135 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088c274-7ccf-447b-8027-e200e99caad2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:47.028170 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-1000bc22-5e52-4955-a917-eaad09292cf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:47.028170 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:47.028170 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:47.028170 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:47.028170 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:47.028170 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:47.028170 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:47.028170 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:47.028170 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:47.028170 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:47.028170 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:47.028170 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:47.028170 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:47.028170 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:47.028170 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:47.028170 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:47.029186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:47.029186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:47.029186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:47.029186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:47.029186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:47.029186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:47.029186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:47.029186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:47.029186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:47.029186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:47.029186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:47.029186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:47.029186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:47.029186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:47.029186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:47.029186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:47.029186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:47.034975 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:47.034975 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:47.034975 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:47.034975 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 224/449] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:47.035285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b088c274-7ccf-447b-8027-e200e99caad2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:47.035370 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000bc22-5e52-4955-a917-eaad09292cf0 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:47.035463 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-b088c274-7ccf-447b-8027-e200e99caad2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 13 12:57:47.035463 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:47.035463 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:47.035463 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:47.035463 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:47.035463 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:47.035463 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:47.035463 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:47.035463 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:47.035463 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:47.035463 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:47.035463 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:47.035463 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:47.035463 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:47.035463 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:47.035463 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:47.036521 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:47.036521 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:47.036521 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:47.036521 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:47.036521 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:47.036521 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:47.036521 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:47.036521 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:47.036521 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:47.036521 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:47.036521 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:47.036521 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:47.036521 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:47.036521 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:47.036521 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:47.036521 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:47.037488 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:47.037488 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:47.037488 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:47.037488 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:47.037488 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:47.037488 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:47.037488 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:47.037488 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:47.037488 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:47.037488 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:47.037488 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 13 12:57:47.037488 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:47.038364 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 226/450] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Oct 13 12:57:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:47.039262 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088c274-7ccf-447b-8027-e200e99caad2 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:47.054599 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:47.093150 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:47.134380 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000bc22-5e52-4955-a917-eaad09292cf0 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:47.137645 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088c274-7ccf-447b-8027-e200e99caad2 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:47.167916 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e92eed-1459-4a4e-b592-e90f7eab3bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:47.168531 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e92eed-1459-4a4e-b592-e90f7eab3bf1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:47.168809 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e92eed-1459-4a4e-b592-e90f7eab3bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:47.169079 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e92eed-1459-4a4e-b592-e90f7eab3bf1 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:47.169763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e92eed-1459-4a4e-b592-e90f7eab3bf1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:47.169862 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d840d6-3b75-429e-8950-8cab20469e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:47.170070 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e92eed-1459-4a4e-b592-e90f7eab3bf1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:47.170391 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d840d6-3b75-429e-8950-8cab20469e45 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:47.170667 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d840d6-3b75-429e-8950-8cab20469e45 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:47.170935 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d840d6-3b75-429e-8950-8cab20469e45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:47.171556 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e92eed-1459-4a4e-b592-e90f7eab3bf1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:47.180642 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d840d6-3b75-429e-8950-8cab20469e45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:47.182052 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d840d6-3b75-429e-8950-8cab20469e45 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:47.182581 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 225/451] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 13 12:57:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:47.183924 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e92eed-1459-4a4e-b592-e90f7eab3bf1 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:47.197600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-75d840d6-3b75-429e-8950-8cab20469e45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:47.197600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:47.197600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:47.197600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:47.197600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:47.197600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:47.197600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:47.197600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:47.197600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:47.197600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:47.197600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:47.197600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:47.197600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:47.197600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:47.197600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:47.197600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:47.198780 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:47.198780 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:47.198780 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:47.198780 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:47.198780 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:47.198780 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:47.198780 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:47.198780 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:47.198780 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:47.198780 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:47.198780 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:47.198780 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:47.198780 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:47.198780 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:47.198780 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:47.198780 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:47.198780 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:47.200297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:47.200297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:47.200297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:47.200297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 227/452] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:47.200297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d840d6-3b75-429e-8950-8cab20469e45 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:47.263159 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:47.283646 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e92eed-1459-4a4e-b592-e90f7eab3bf1 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:47.296612 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d840d6-3b75-429e-8950-8cab20469e45 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:47.299070 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:47.317603 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b70970-3777-4cb7-bba0-fcb35609eb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:47.318224 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b70970-3777-4cb7-bba0-fcb35609eb24 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:47.318506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b70970-3777-4cb7-bba0-fcb35609eb24 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:47.318776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b70970-3777-4cb7-bba0-fcb35609eb24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:47.329702 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b70970-3777-4cb7-bba0-fcb35609eb24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:47.331128 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af6928-2e87-4205-b69a-97eda9264266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:47.331270 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b70970-3777-4cb7-bba0-fcb35609eb24 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:47.331721 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af6928-2e87-4205-b69a-97eda9264266 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:47.331997 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af6928-2e87-4205-b69a-97eda9264266 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:47.332263 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af6928-2e87-4205-b69a-97eda9264266 None admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:47.332894 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af6928-2e87-4205-b69a-97eda9264266 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:47.333192 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af6928-2e87-4205-b69a-97eda9264266 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:47.334539 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af6928-2e87-4205-b69a-97eda9264266 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:47.344797 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-a6b70970-3777-4cb7-bba0-fcb35609eb24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:47.344797 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:47.344797 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:47.344797 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:47.344797 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:47.344797 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:47.344797 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:47.344797 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:47.344797 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:47.344797 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:47.344797 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:47.344797 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:47.344797 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:47.344797 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:47.344797 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:47.344797 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:47.345885 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:47.345885 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:47.345885 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:47.345885 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:47.345885 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:47.345885 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:47.345885 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:47.345885 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:47.345885 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:47.345885 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:47.345885 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:47.345885 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:47.345885 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:47.345885 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:47.345885 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:47.345885 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:47.345885 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:47.346726 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:47.346726 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:47.346726 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:47.346726 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 226/453] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:47.346992 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b70970-3777-4cb7-bba0-fcb35609eb24 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:47.347491 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 228/454] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:47.348416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af6928-2e87-4205-b69a-97eda9264266 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:47.443039 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b70970-3777-4cb7-bba0-fcb35609eb24 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:47.445593 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af6928-2e87-4205-b69a-97eda9264266 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:47.471366 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:47.476566 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e7bf31-120c-43ef-927b-e3c9521537f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:47.477156 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7bf31-120c-43ef-927b-e3c9521537f3 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:47.477430 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7bf31-120c-43ef-927b-e3c9521537f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:47.477696 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7bf31-120c-43ef-927b-e3c9521537f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:47.478403 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e7bf31-120c-43ef-927b-e3c9521537f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:47.478703 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e7bf31-120c-43ef-927b-e3c9521537f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:47.480694 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e7bf31-120c-43ef-927b-e3c9521537f3 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:47.480795 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aee0f1-d64e-4dba-96e3-87c5b48b88c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:47.481352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aee0f1-d64e-4dba-96e3-87c5b48b88c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:47.481632 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aee0f1-d64e-4dba-96e3-87c5b48b88c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:47.481906 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aee0f1-d64e-4dba-96e3-87c5b48b88c6 None admin] PATH_INFO: `/v3/projects/02308892635844b088ecdcb5413a21a0/users/526000341fd146db8c661f1fd14d4801/roles/b493ef4ee9d44f449e08251ae75a77d8` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:47.488333 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 227/455] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Oct 13 12:57:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:47.490612 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e7bf31-120c-43ef-927b-e3c9521537f3 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:47.497391 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aee0f1-d64e-4dba-96e3-87c5b48b88c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=02308892635844b088ecdcb5413a21a0, user_id=526000341fd146db8c661f1fd14d4801, role_id=b493ef4ee9d44f449e08251ae75a77d8)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:47.498852 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aee0f1-d64e-4dba-96e3-87c5b48b88c6 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:47.511471 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 229/456] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 13 12:57:47 2022] PUT /identity/v3/projects/02308892635844b088ecdcb5413a21a0/users/526000341fd146db8c661f1fd14d4801/roles/b493ef4ee9d44f449e08251ae75a77d8 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 12:57:47.512259 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aee0f1-d64e-4dba-96e3-87c5b48b88c6 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:47.521806 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:47.607752 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e7bf31-120c-43ef-927b-e3c9521537f3 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:47.634628 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aee0f1-d64e-4dba-96e3-87c5b48b88c6 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:47.647256 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e11d91-5373-4bfa-baee-53233a059966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:47.647864 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e11d91-5373-4bfa-baee-53233a059966 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:47.648142 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e11d91-5373-4bfa-baee-53233a059966 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:47.648409 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e11d91-5373-4bfa-baee-53233a059966 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:47.649076 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e11d91-5373-4bfa-baee-53233a059966 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:47.649374 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e11d91-5373-4bfa-baee-53233a059966 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:47.650769 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e11d91-5373-4bfa-baee-53233a059966 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:47.663712 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 228/457] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:47.664573 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e11d91-5373-4bfa-baee-53233a059966 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:47.679890 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7234d9-e856-45c9-8e06-f1ab609fe69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:47.680537 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7234d9-e856-45c9-8e06-f1ab609fe69e None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:47.680912 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7234d9-e856-45c9-8e06-f1ab609fe69e None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:47.681087 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7234d9-e856-45c9-8e06-f1ab609fe69e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:47.690504 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7234d9-e856-45c9-8e06-f1ab609fe69e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:47.692117 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7234d9-e856-45c9-8e06-f1ab609fe69e None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:47.703664 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-ee7234d9-e856-45c9-8e06-f1ab609fe69e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 13 12:57:47.703664 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:47.703664 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:47.703664 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:47.703664 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:47.703664 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:47.703664 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:47.703664 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:47.703664 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:47.703664 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:47.703664 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:47.703664 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:47.703664 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 13 12:57:47.703664 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 13 12:57:47.703664 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 13 12:57:47.703664 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 13 12:57:47.704769 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:47.704769 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:47.704769 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 13 12:57:47.704769 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:57:47.704769 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 13 12:57:47.704769 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:57:47.704769 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:47.704769 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:47.704769 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:47.704769 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:47.704769 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:47.704769 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:47.704769 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:47.704769 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:47.704769 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:47.704769 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:47.704769 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:47.704769 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:47.705673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:47.705673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:47.705673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 13 12:57:47.705673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 13 12:57:47.705673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 13 12:57:47.705673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 13 12:57:47.705673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 13 12:57:47.705673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 13 12:57:47.705673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 13 12:57:47.705673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:47.705673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 230/458] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:57:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:47.705673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7234d9-e856-45c9-8e06-f1ab609fe69e None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:47.744252 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:47.772029 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:47.783891 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e11d91-5373-4bfa-baee-53233a059966 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:47.821730 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bde454-0f15-420d-919c-62c38d1da991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:47.822336 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bde454-0f15-420d-919c-62c38d1da991 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:47.822613 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bde454-0f15-420d-919c-62c38d1da991 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:47.822881 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bde454-0f15-420d-919c-62c38d1da991 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:47.823473 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7234d9-e856-45c9-8e06-f1ab609fe69e None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:47.823573 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bde454-0f15-420d-919c-62c38d1da991 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:47.823884 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bde454-0f15-420d-919c-62c38d1da991 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:47.825259 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bde454-0f15-420d-919c-62c38d1da991 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:47.836862 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 229/459] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:47.838673 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bde454-0f15-420d-919c-62c38d1da991 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:47.860383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fbce38-3135-41fa-b2f3-0933ed272415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:47.860972 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbce38-3135-41fa-b2f3-0933ed272415 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:47.861254 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbce38-3135-41fa-b2f3-0933ed272415 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:47.861524 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbce38-3135-41fa-b2f3-0933ed272415 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:47.872573 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fbce38-3135-41fa-b2f3-0933ed272415 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:47.874038 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fbce38-3135-41fa-b2f3-0933ed272415 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:47.891071 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-16fbce38-3135-41fa-b2f3-0933ed272415 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:47.891071 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:47.891071 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:47.891071 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:47.891071 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:47.891071 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:47.891071 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:47.891071 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:47.891071 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:47.891071 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:47.891071 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:47.891071 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:47.891071 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:47.891071 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:47.891071 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:47.891071 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:47.891981 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:47.891981 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:47.891981 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:47.891981 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:47.891981 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:47.891981 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:47.891981 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:47.891981 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:47.891981 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:47.891981 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:47.891981 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:47.891981 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:47.891981 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:47.891981 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:47.891981 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:47.891981 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:47.891981 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:47.893403 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:47.893403 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:47.893403 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:47.893403 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 231/460] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:47.893403 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fbce38-3135-41fa-b2f3-0933ed272415 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:47.953650 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bde454-0f15-420d-919c-62c38d1da991 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:47.988518 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ac0d1-a683-4a28-a7bc-d21b2aa78b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:47.989518 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ac0d1-a683-4a28-a7bc-d21b2aa78b30 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:47.989997 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ac0d1-a683-4a28-a7bc-d21b2aa78b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:47.990463 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ac0d1-a683-4a28-a7bc-d21b2aa78b30 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:47.995754 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:47.998518 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ac0d1-a683-4a28-a7bc-d21b2aa78b30 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:48.000013 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ac0d1-a683-4a28-a7bc-d21b2aa78b30 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:48.008443 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fbce38-3135-41fa-b2f3-0933ed272415 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:48.012399 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-4c2ac0d1-a683-4a28-a7bc-d21b2aa78b30 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 13 12:57:48.012399 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:48.012399 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:48.012399 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:48.012399 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:48.012399 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:48.012399 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:48.012399 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:48.012399 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:48.012399 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:48.012399 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:48.012399 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:48.012399 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 13 12:57:48.012399 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 13 12:57:48.012399 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 13 12:57:48.012399 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 13 12:57:48.013288 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:48.013288 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:48.013288 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 13 12:57:48.013288 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:57:48.013288 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 13 12:57:48.013288 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:57:48.013288 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:48.013288 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:48.013288 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:48.013288 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:48.013288 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:48.013288 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:48.013288 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:48.013288 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:48.013288 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:48.013288 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:48.013288 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:48.013288 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:48.014753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:48.014753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:48.014753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 13 12:57:48.014753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 13 12:57:48.014753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 13 12:57:48.014753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 13 12:57:48.014753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 13 12:57:48.014753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 13 12:57:48.014753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 13 12:57:48.014753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:48.014753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 230/461] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:57:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:48.014753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ac0d1-a683-4a28-a7bc-d21b2aa78b30 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:48.032137 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:48.048144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f884c1-8761-4ba8-a0f1-08c334365131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:48.048830 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f884c1-8761-4ba8-a0f1-08c334365131 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:48.049110 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f884c1-8761-4ba8-a0f1-08c334365131 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:48.049393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f884c1-8761-4ba8-a0f1-08c334365131 None admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:48.050109 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f884c1-8761-4ba8-a0f1-08c334365131 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:48.050438 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f884c1-8761-4ba8-a0f1-08c334365131 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:48.052145 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f884c1-8761-4ba8-a0f1-08c334365131 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:48.074624 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 232/462] 10.222.0.23 () {56 vars in 1137 bytes} [Thu Oct 13 12:57:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:48.074624 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f884c1-8761-4ba8-a0f1-08c334365131 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:48.125027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ac0d1-a683-4a28-a7bc-d21b2aa78b30 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:48.158526 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f218887-897d-4844-82fb-32a0035ca7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:48.159182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f218887-897d-4844-82fb-32a0035ca7cd None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:48.159462 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f218887-897d-4844-82fb-32a0035ca7cd None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:48.159732 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f218887-897d-4844-82fb-32a0035ca7cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:48.166517 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f218887-897d-4844-82fb-32a0035ca7cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:48.168539 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f218887-897d-4844-82fb-32a0035ca7cd None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:48.177990 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f884c1-8761-4ba8-a0f1-08c334365131 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:48.179434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-3f218887-897d-4844-82fb-32a0035ca7cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:57:48.179434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:48.179434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:48.179434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:48.179434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:48.179434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:48.179434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:48.179434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:48.179434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:48.179434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:48.179434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:48.179434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:48.179434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 13 12:57:48.179434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 13 12:57:48.179434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 13 12:57:48.180454 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 13 12:57:48.180454 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:48.180454 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:48.180454 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:48.180454 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:48.180454 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:48.180454 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:48.180454 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:48.180454 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:48.180454 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:48.180454 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:48.180454 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:48.180454 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:48.180454 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:48.180454 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:48.180454 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:48.180454 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:48.181676 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 13 12:57:48.181676 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 13 12:57:48.181676 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 13 12:57:48.181676 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 13 12:57:48.181676 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 13 12:57:48.181676 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 13 12:57:48.181676 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:57:48.181676 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:48.181676 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 231/463] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:57:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:48.181676 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f218887-897d-4844-82fb-32a0035ca7cd None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:48.220214 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349ea84-dd06-47a6-89cc-509a8c6c4db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:48.220833 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349ea84-dd06-47a6-89cc-509a8c6c4db1 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:48.221264 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349ea84-dd06-47a6-89cc-509a8c6c4db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:48.221547 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349ea84-dd06-47a6-89cc-509a8c6c4db1 None admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:48.222200 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c349ea84-dd06-47a6-89cc-509a8c6c4db1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:48.222507 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c349ea84-dd06-47a6-89cc-509a8c6c4db1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:48.223903 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c349ea84-dd06-47a6-89cc-509a8c6c4db1 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:48.240222 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 233/464] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:48.240222 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349ea84-dd06-47a6-89cc-509a8c6c4db1 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:48.245321 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:48.263615 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:48.300862 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f218887-897d-4844-82fb-32a0035ca7cd None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:48.337914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ae744-219d-4006-a442-e7ba08ba509a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:48.338514 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ae744-219d-4006-a442-e7ba08ba509a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:48.338824 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ae744-219d-4006-a442-e7ba08ba509a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:48.339118 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ae744-219d-4006-a442-e7ba08ba509a None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:48.340445 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9ae744-219d-4006-a442-e7ba08ba509a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:48.340756 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9ae744-219d-4006-a442-e7ba08ba509a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:48.341313 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349ea84-dd06-47a6-89cc-509a8c6c4db1 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:48.342186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9ae744-219d-4006-a442-e7ba08ba509a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:48.360911 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 232/465] 10.222.0.23 () {56 vars in 1137 bytes} [Thu Oct 13 12:57:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:48.362086 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ae744-219d-4006-a442-e7ba08ba509a None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:48.380030 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8165693-5631-4f95-a680-4fefa83f4f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:48.380571 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8165693-5631-4f95-a680-4fefa83f4f12 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:48.380865 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8165693-5631-4f95-a680-4fefa83f4f12 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:48.381119 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8165693-5631-4f95-a680-4fefa83f4f12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:48.455627 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8165693-5631-4f95-a680-4fefa83f4f12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:48.457150 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8165693-5631-4f95-a680-4fefa83f4f12 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:48.466085 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:48.472588 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ae744-219d-4006-a442-e7ba08ba509a None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:48.500621 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:48.509513 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f0159-d283-4200-a1f6-78a21fd2b20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:48.510118 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f0159-d283-4200-a1f6-78a21fd2b20a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:48.510403 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f0159-d283-4200-a1f6-78a21fd2b20a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:48.510677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f0159-d283-4200-a1f6-78a21fd2b20a None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:48.511930 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f0159-d283-4200-a1f6-78a21fd2b20a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:48.512234 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f0159-d283-4200-a1f6-78a21fd2b20a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:48.513647 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f0159-d283-4200-a1f6-78a21fd2b20a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:48.526297 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 233/466] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 13 12:57:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:48.527626 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f0159-d283-4200-a1f6-78a21fd2b20a None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:48.569558 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 234/467] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 13 12:57:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 13 12:57:48.570093 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8165693-5631-4f95-a680-4fefa83f4f12 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:48.642058 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f0159-d283-4200-a1f6-78a21fd2b20a None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:48.670090 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8165693-5631-4f95-a680-4fefa83f4f12 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:48.676158 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c99b87-9d9b-45ea-8d64-145e90ee2ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:48.676783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c99b87-9d9b-45ea-8d64-145e90ee2ac0 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:48.677066 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c99b87-9d9b-45ea-8d64-145e90ee2ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:48.677338 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c99b87-9d9b-45ea-8d64-145e90ee2ac0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:48.678265 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:48.685923 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c99b87-9d9b-45ea-8d64-145e90ee2ac0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:48.687636 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c99b87-9d9b-45ea-8d64-145e90ee2ac0 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:48.698460 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-01c99b87-9d9b-45ea-8d64-145e90ee2ac0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 13 12:57:48.698460 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:48.698460 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:48.698460 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:48.698460 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:48.698460 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:48.698460 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:48.698460 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:48.698460 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:48.698460 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:48.698460 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:48.698460 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:48.698460 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 13 12:57:48.698460 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 13 12:57:48.698460 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 13 12:57:48.698460 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 13 12:57:48.699951 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:48.699951 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:48.699951 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 13 12:57:48.699951 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:57:48.699951 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 13 12:57:48.699951 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:57:48.699951 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:48.699951 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:48.699951 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:48.699951 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:48.699951 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:48.699951 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:48.699951 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:48.699951 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:48.699951 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:48.699951 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:48.699951 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:48.699951 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:48.700845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:48.700845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:48.700845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 13 12:57:48.700845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 13 12:57:48.700845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 13 12:57:48.700845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 13 12:57:48.700845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 13 12:57:48.700845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 13 12:57:48.700845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 13 12:57:48.700845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:48.705034 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 234/468] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Oct 13 12:57:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:48.705780 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c99b87-9d9b-45ea-8d64-145e90ee2ac0 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:48.710406 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532b402-5bf4-488c-adc0-f2ea10fabc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:48.711011 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532b402-5bf4-488c-adc0-f2ea10fabc93 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:48.711320 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532b402-5bf4-488c-adc0-f2ea10fabc93 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:48.711585 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532b402-5bf4-488c-adc0-f2ea10fabc93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:48.757228 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:48.785847 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1532b402-5bf4-488c-adc0-f2ea10fabc93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:48.787345 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1532b402-5bf4-488c-adc0-f2ea10fabc93 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:48.812330 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c99b87-9d9b-45ea-8d64-145e90ee2ac0 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:48.848151 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7dd07-71ea-417d-90f3-a38ac9e46b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:48.848733 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7dd07-71ea-417d-90f3-a38ac9e46b7b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:48.849007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7dd07-71ea-417d-90f3-a38ac9e46b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:48.849274 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7dd07-71ea-417d-90f3-a38ac9e46b7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:48.850119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b7dd07-71ea-417d-90f3-a38ac9e46b7b None admin] RBAC: Adding query filter params (role.id=b493ef4ee9d44f449e08251ae75a77d8, scope.project.id=02308892635844b088ecdcb5413a21a0, user.id=12700d985a724595af630c81e63f86d7) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:48.850391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b7dd07-71ea-417d-90f3-a38ac9e46b7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:48.851780 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b7dd07-71ea-417d-90f3-a38ac9e46b7b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:48.863091 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 235/469] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 13 12:57:48 2022] GET /identity/v3/role_assignments?user.id=12700d985a724595af630c81e63f86d7&scope.project.id=02308892635844b088ecdcb5413a21a0&role.id=b493ef4ee9d44f449e08251ae75a77d8 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:48.863091 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7dd07-71ea-417d-90f3-a38ac9e46b7b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:48.900148 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 235/470] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 13 12:57:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 13 12:57:48.901070 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532b402-5bf4-488c-adc0-f2ea10fabc93 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:48.916859 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:48.976196 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7dd07-71ea-417d-90f3-a38ac9e46b7b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:48.991901 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:49.018404 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a8532c-2b25-4758-a52f-3c1468570df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:49.019131 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a8532c-2b25-4758-a52f-3c1468570df1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:49.019420 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a8532c-2b25-4758-a52f-3c1468570df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:49.019692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a8532c-2b25-4758-a52f-3c1468570df1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:49.031602 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a8532c-2b25-4758-a52f-3c1468570df1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:49.033080 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a8532c-2b25-4758-a52f-3c1468570df1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:49.049161 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-11a8532c-2b25-4758-a52f-3c1468570df1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:49.049161 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:49.049161 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:49.049161 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:49.049161 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:49.049161 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:49.049161 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:49.049161 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:49.049161 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:49.049161 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:49.049161 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:49.049161 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:49.049161 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:49.049161 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:49.049161 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:49.049161 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:49.050115 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:49.050115 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:49.050115 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:49.050115 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:49.050115 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:49.050115 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:49.050115 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:49.050115 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:49.050115 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:49.050115 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:49.050115 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:49.050115 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:49.050115 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:49.050115 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:49.050115 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:49.050115 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:49.050115 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:49.050962 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:49.050962 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:49.050962 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:49.050962 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 236/471] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:49.051631 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a8532c-2b25-4758-a52f-3c1468570df1 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:49.069425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532b402-5bf4-488c-adc0-f2ea10fabc93 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:49.111353 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4906a909-e4ac-4bb9-b017-9b0c7a739562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:49.111920 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906a909-e4ac-4bb9-b017-9b0c7a739562 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:49.112198 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906a909-e4ac-4bb9-b017-9b0c7a739562 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:49.112789 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906a909-e4ac-4bb9-b017-9b0c7a739562 None admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:49.113524 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4906a909-e4ac-4bb9-b017-9b0c7a739562 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:49.113832 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4906a909-e4ac-4bb9-b017-9b0c7a739562 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:49.116185 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4906a909-e4ac-4bb9-b017-9b0c7a739562 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:49.134401 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 236/472] 10.222.0.23 () {56 vars in 1139 bytes} [Thu Oct 13 12:57:48 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:49.135285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4906a909-e4ac-4bb9-b017-9b0c7a739562 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:49.176361 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a8532c-2b25-4758-a52f-3c1468570df1 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:49.212969 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:49.215766 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8e09a-dfc9-468d-8e5e-000d88a26297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:49.217187 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8e09a-dfc9-468d-8e5e-000d88a26297 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:49.217647 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8e09a-dfc9-468d-8e5e-000d88a26297 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:49.218506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8e09a-dfc9-468d-8e5e-000d88a26297 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:49.235594 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8e09a-dfc9-468d-8e5e-000d88a26297 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:49.237377 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8e09a-dfc9-468d-8e5e-000d88a26297 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:49.253105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-21b8e09a-dfc9-468d-8e5e-000d88a26297 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:49.253105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:49.253105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:49.253105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:49.253105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:49.253105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:49.253105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:49.253105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:49.253105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:49.253105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:49.253105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:49.253105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:49.253105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:49.253105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:49.253105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:49.253105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:49.254084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:49.254084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:49.254084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:49.254084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:49.254084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:49.254084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:49.254084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:49.254084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:49.254084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:49.254084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:49.254084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:49.254084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:49.254084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:49.254084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:49.254084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:49.254084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:49.254084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:49.254938 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:49.254938 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:49.254938 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:49.254938 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 237/473] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:49.255964 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8e09a-dfc9-468d-8e5e-000d88a26297 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:49.261719 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4906a909-e4ac-4bb9-b017-9b0c7a739562 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:49.275642 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:49.311878 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6947199-d494-4016-8435-7468d64d9f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:49.312919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6947199-d494-4016-8435-7468d64d9f3c None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:49.313341 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6947199-d494-4016-8435-7468d64d9f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:49.313766 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6947199-d494-4016-8435-7468d64d9f3c None admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:49.315345 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6947199-d494-4016-8435-7468d64d9f3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:49.316343 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6947199-d494-4016-8435-7468d64d9f3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:49.318510 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6947199-d494-4016-8435-7468d64d9f3c None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:49.331982 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 237/474] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:49.333074 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6947199-d494-4016-8435-7468d64d9f3c None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:49.377510 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8e09a-dfc9-468d-8e5e-000d88a26297 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:49.419635 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69bce0d-de3a-45f8-9bdd-78f45ef8e33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:49.420588 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bce0d-de3a-45f8-9bdd-78f45ef8e33e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:49.421030 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bce0d-de3a-45f8-9bdd-78f45ef8e33e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:49.421462 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bce0d-de3a-45f8-9bdd-78f45ef8e33e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:49.443956 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6947199-d494-4016-8435-7468d64d9f3c None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:49.468354 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:49.481578 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422e161-c24a-4a69-b9b3-cfa23ff5a40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:49.482343 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422e161-c24a-4a69-b9b3-cfa23ff5a40b None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:49.482724 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422e161-c24a-4a69-b9b3-cfa23ff5a40b None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:49.483128 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422e161-c24a-4a69-b9b3-cfa23ff5a40b None admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:49.483863 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c422e161-c24a-4a69-b9b3-cfa23ff5a40b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:49.484269 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c422e161-c24a-4a69-b9b3-cfa23ff5a40b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:49.485730 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c422e161-c24a-4a69-b9b3-cfa23ff5a40b None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:49.498103 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 238/475] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:49.499260 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422e161-c24a-4a69-b9b3-cfa23ff5a40b None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:49.506790 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69bce0d-de3a-45f8-9bdd-78f45ef8e33e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:49.509363 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69bce0d-de3a-45f8-9bdd-78f45ef8e33e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:49.532059 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:49.607817 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422e161-c24a-4a69-b9b3-cfa23ff5a40b None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:49.640677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 238/476] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 13 12:57:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 13 12:57:49.640677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69bce0d-de3a-45f8-9bdd-78f45ef8e33e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:49.652267 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f97a8-77ae-4487-9d4a-3ccc90a03b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:49.652916 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f97a8-77ae-4487-9d4a-3ccc90a03b14 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:49.653530 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f97a8-77ae-4487-9d4a-3ccc90a03b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:49.653530 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f97a8-77ae-4487-9d4a-3ccc90a03b14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:49.661151 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f97a8-77ae-4487-9d4a-3ccc90a03b14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:49.662703 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f97a8-77ae-4487-9d4a-3ccc90a03b14 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:49.677234 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-0a7f97a8-77ae-4487-9d4a-3ccc90a03b14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:57:49.677234 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:49.677234 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:49.677234 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:49.677234 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:49.677234 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:49.677234 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:49.677234 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:49.677234 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:49.677234 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:49.677234 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:49.677234 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:49.677234 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 13 12:57:49.677234 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 13 12:57:49.677234 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 13 12:57:49.678535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 13 12:57:49.678535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:49.678535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:49.678535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:49.678535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:49.678535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:49.678535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:49.678535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:49.678535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:49.678535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:49.678535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:49.678535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:49.678535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:49.678535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:49.678535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:49.678535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:49.678535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:49.679391 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 13 12:57:49.679391 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 13 12:57:49.679391 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 13 12:57:49.679391 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 13 12:57:49.679391 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 13 12:57:49.679391 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 13 12:57:49.679391 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:57:49.679391 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:49.683860 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 239/477] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:57:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:49.684440 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f97a8-77ae-4487-9d4a-3ccc90a03b14 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:49.735569 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:49.749831 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69bce0d-de3a-45f8-9bdd-78f45ef8e33e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:49.766498 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:49.785184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d66cf3-89f2-41a6-83c6-cb5e0475fab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:49.785822 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d66cf3-89f2-41a6-83c6-cb5e0475fab4 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:49.786101 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d66cf3-89f2-41a6-83c6-cb5e0475fab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:49.786376 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d66cf3-89f2-41a6-83c6-cb5e0475fab4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:49.792929 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d66cf3-89f2-41a6-83c6-cb5e0475fab4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:49.794379 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d66cf3-89f2-41a6-83c6-cb5e0475fab4 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:49.795359 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f97a8-77ae-4487-9d4a-3ccc90a03b14 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:49.805502 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-b5d66cf3-89f2-41a6-83c6-cb5e0475fab4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:57:49.805502 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:49.805502 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:49.805502 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:49.805502 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:49.805502 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:49.805502 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:49.805502 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:49.805502 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:49.805502 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:49.805502 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:49.805502 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:49.805502 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 13 12:57:49.805502 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 13 12:57:49.805502 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 13 12:57:49.806343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 13 12:57:49.806343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:49.806343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:49.806343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:49.806343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:49.806343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:49.806343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:49.806343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:49.806343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:49.806343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:49.806343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:49.806343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:49.806343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:49.806343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:49.806343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:49.806343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:49.806343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:49.807245 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 13 12:57:49.807245 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 13 12:57:49.807245 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 13 12:57:49.807245 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 13 12:57:49.807245 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 13 12:57:49.807245 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 13 12:57:49.807245 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:57:49.807245 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:49.811364 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 239/478] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:57:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:49.812846 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946eb379-1ff0-4cbc-82e0-ff3336588186 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:49.813134 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946eb379-1ff0-4cbc-82e0-ff3336588186 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:49.813415 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946eb379-1ff0-4cbc-82e0-ff3336588186 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:49.814712 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 240/479] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:49.815374 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946eb379-1ff0-4cbc-82e0-ff3336588186 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:49.834467 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf41e9-abbb-465e-95c5-3a8055726ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:49.835685 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf41e9-abbb-465e-95c5-3a8055726ae0 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:49.836102 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf41e9-abbb-465e-95c5-3a8055726ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:49.836510 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf41e9-abbb-465e-95c5-3a8055726ae0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:49.848024 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf41e9-abbb-465e-95c5-3a8055726ae0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:49.849764 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf41e9-abbb-465e-95c5-3a8055726ae0 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:49.864600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-c0cf41e9-abbb-465e-95c5-3a8055726ae0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:49.864600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:49.864600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:49.864600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:49.864600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:49.864600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:49.864600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:49.864600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:49.864600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:49.864600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:49.864600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:49.864600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:49.864600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:49.864600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:49.864600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:49.864600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:49.865673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:49.865673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:49.865673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:49.865673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:49.865673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:49.865673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:49.865673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:49.865673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:49.865673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:49.865673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:49.865673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:49.865673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:49.865673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:49.865673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:49.865673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:49.865673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:49.865673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:49.866693 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:49.866693 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:49.866693 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:49.868109 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 240/480] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:49.869037 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf41e9-abbb-465e-95c5-3a8055726ae0 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:49.937180 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946eb379-1ff0-4cbc-82e0-ff3336588186 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:49.972322 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f4ff26-9e32-4492-9569-648c3965f2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:49.973054 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4ff26-9e32-4492-9569-648c3965f2f6 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:49.973339 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4ff26-9e32-4492-9569-648c3965f2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:49.973605 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4ff26-9e32-4492-9569-648c3965f2f6 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:49.974297 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f4ff26-9e32-4492-9569-648c3965f2f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:49.974604 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f4ff26-9e32-4492-9569-648c3965f2f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:49.975322 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:49.976467 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f4ff26-9e32-4492-9569-648c3965f2f6 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:49.982645 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf41e9-abbb-465e-95c5-3a8055726ae0 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:49.988892 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 241/481] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 13 12:57:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:49.989836 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4bf5ec-265b-4147-95f2-917fb175b03d None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:49.990121 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4bf5ec-265b-4147-95f2-917fb175b03d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:49.990397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4bf5ec-265b-4147-95f2-917fb175b03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:50.008014 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6b4bf5ec-265b-4147-95f2-917fb175b03d None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:50.009245 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6b4bf5ec-265b-4147-95f2-917fb175b03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:50.023357 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473739c-7f30-4597-9867-9c2c5fcd02c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:50.024561 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473739c-7f30-4597-9867-9c2c5fcd02c8 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:50.025347 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473739c-7f30-4597-9867-9c2c5fcd02c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:50.025806 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473739c-7f30-4597-9867-9c2c5fcd02c8 None admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:50.027398 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4473739c-7f30-4597-9867-9c2c5fcd02c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:50.028148 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4473739c-7f30-4597-9867-9c2c5fcd02c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:50.031176 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4473739c-7f30-4597-9867-9c2c5fcd02c8 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:50.038549 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:50.047897 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 241/482] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 13 12:57:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:50.048900 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473739c-7f30-4597-9867-9c2c5fcd02c8 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:50.079754 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 242/483] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:50.080589 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4bf5ec-265b-4147-95f2-917fb175b03d None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:50.191136 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4bf5ec-265b-4147-95f2-917fb175b03d None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:50.194218 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473739c-7f30-4597-9867-9c2c5fcd02c8 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:50.232322 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825b86a-7791-4c54-98db-02082299d8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:50.233190 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825b86a-7791-4c54-98db-02082299d8ed None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:50.233528 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825b86a-7791-4c54-98db-02082299d8ed None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:50.233808 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825b86a-7791-4c54-98db-02082299d8ed None admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:50.234458 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825b86a-7791-4c54-98db-02082299d8ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:50.234807 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825b86a-7791-4c54-98db-02082299d8ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:50.236236 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825b86a-7791-4c54-98db-02082299d8ed None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:50.238137 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9633fdc1-4c73-4eb3-aed0-c02f30ee28a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:50.239554 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633fdc1-4c73-4eb3-aed0-c02f30ee28a8 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:50.240048 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633fdc1-4c73-4eb3-aed0-c02f30ee28a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:50.240521 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633fdc1-4c73-4eb3-aed0-c02f30ee28a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:50.241496 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9633fdc1-4c73-4eb3-aed0-c02f30ee28a8 None admin] RBAC: Adding query filter params (role.id=b493ef4ee9d44f449e08251ae75a77d8, scope.project.id=02308892635844b088ecdcb5413a21a0, user.id=45ce234c5b4643b2ac6341e0928f3830) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:50.241997 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9633fdc1-4c73-4eb3-aed0-c02f30ee28a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:50.244279 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9633fdc1-4c73-4eb3-aed0-c02f30ee28a8 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:50.252975 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 242/484] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:50.252975 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825b86a-7791-4c54-98db-02082299d8ed None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:50.256939 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 243/485] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 13 12:57:50 2022] GET /identity/v3/role_assignments?user.id=45ce234c5b4643b2ac6341e0928f3830&scope.project.id=02308892635844b088ecdcb5413a21a0&role.id=b493ef4ee9d44f449e08251ae75a77d8 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:50.258064 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9633fdc1-4c73-4eb3-aed0-c02f30ee28a8 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:50.272903 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:50.297522 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:50.408177 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825b86a-7791-4c54-98db-02082299d8ed None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:50.411782 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9633fdc1-4c73-4eb3-aed0-c02f30ee28a8 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:50.457425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486e0ab-d8d4-42b6-b3c8-172b605dc8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:50.458074 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486e0ab-d8d4-42b6-b3c8-172b605dc8a3 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:50.458403 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486e0ab-d8d4-42b6-b3c8-172b605dc8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:50.458682 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486e0ab-d8d4-42b6-b3c8-172b605dc8a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:50.459460 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0486e0ab-d8d4-42b6-b3c8-172b605dc8a3 None admin] RBAC: Adding query filter params (role.id=b493ef4ee9d44f449e08251ae75a77d8, scope.project.id=02308892635844b088ecdcb5413a21a0, user.id=2398a599165a46adb5718d98c897f238) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:50.459774 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0486e0ab-d8d4-42b6-b3c8-172b605dc8a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:50.461253 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0486e0ab-d8d4-42b6-b3c8-172b605dc8a3 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:50.463009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a81bf-6401-4b7d-934f-f7f307571a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:50.464213 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a81bf-6401-4b7d-934f-f7f307571a59 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:50.464668 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a81bf-6401-4b7d-934f-f7f307571a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:50.465755 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a81bf-6401-4b7d-934f-f7f307571a59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:50.468912 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 243/486] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 13 12:57:50 2022] GET /identity/v3/role_assignments?user.id=2398a599165a46adb5718d98c897f238&scope.project.id=02308892635844b088ecdcb5413a21a0&role.id=b493ef4ee9d44f449e08251ae75a77d8 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:50.470181 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486e0ab-d8d4-42b6-b3c8-172b605dc8a3 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:50.474906 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9a81bf-6401-4b7d-934f-f7f307571a59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:50.476674 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9a81bf-6401-4b7d-934f-f7f307571a59 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:50.491516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-8c9a81bf-6401-4b7d-934f-f7f307571a59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:57:50.491516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:50.491516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:50.491516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:50.491516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:50.491516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:50.491516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:50.491516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:50.491516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:50.491516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:50.491516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:50.491516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:50.491516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 13 12:57:50.491516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 13 12:57:50.491516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 13 12:57:50.492408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 13 12:57:50.492408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:50.492408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:50.492408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:50.492408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:50.492408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:50.492408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:50.492408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:50.492408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:50.492408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:50.492408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:50.492408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:50.492408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:50.492408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:50.492408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:50.492408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:50.492408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:50.493246 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 13 12:57:50.493246 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 13 12:57:50.493246 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 13 12:57:50.493246 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 13 12:57:50.493246 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 13 12:57:50.493246 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 13 12:57:50.493246 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:57:50.493246 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:50.496503 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 244/487] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:57:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:50.502621 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a81bf-6401-4b7d-934f-f7f307571a59 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:50.592935 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:50.614920 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:50.624419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486e0ab-d8d4-42b6-b3c8-172b605dc8a3 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:50.626545 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a81bf-6401-4b7d-934f-f7f307571a59 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:50.668370 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6361b871-37f6-498e-b8d3-922795a96f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:50.669140 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361b871-37f6-498e-b8d3-922795a96f97 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:50.670173 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b08e2c-08ad-4c9a-a579-13405ab02c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:50.670502 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361b871-37f6-498e-b8d3-922795a96f97 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:50.671040 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361b871-37f6-498e-b8d3-922795a96f97 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:50.671291 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b08e2c-08ad-4c9a-a579-13405ab02c90 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:50.671705 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b08e2c-08ad-4c9a-a579-13405ab02c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:50.672129 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b08e2c-08ad-4c9a-a579-13405ab02c90 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:50.672237 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6361b871-37f6-498e-b8d3-922795a96f97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:50.672639 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6361b871-37f6-498e-b8d3-922795a96f97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:50.674182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6361b871-37f6-498e-b8d3-922795a96f97 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:50.685854 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b08e2c-08ad-4c9a-a579-13405ab02c90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:50.689815 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 245/488] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 13 12:57:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:57:50.697275 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-80b08e2c-08ad-4c9a-a579-13405ab02c90 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 13 12:57:50.697275 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:50.697275 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:50.697275 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:50.697275 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:50.697275 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:50.697275 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:50.697275 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:50.697275 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:50.697275 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:50.697275 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:50.697275 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:50.697275 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:50.697275 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:50.697275 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:50.697275 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:50.698172 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:50.698172 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:50.698172 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:50.698172 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:50.698172 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:50.698172 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:50.698172 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:50.698172 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:50.698172 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:50.698172 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:50.698172 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:50.698172 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:50.698172 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:50.698172 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:50.698172 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:50.698172 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:50.699047 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:50.699047 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:50.699047 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:50.699047 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:50.699047 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:50.699047 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:50.699047 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:50.699047 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:50.699047 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:50.699047 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:50.699047 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 13 12:57:50.699047 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:50.699047 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 244/489] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Oct 13 12:57:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:50.700869 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6361b871-37f6-498e-b8d3-922795a96f97 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:50.707026 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b08e2c-08ad-4c9a-a579-13405ab02c90 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:50.834125 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6361b871-37f6-498e-b8d3-922795a96f97 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:50.834896 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b08e2c-08ad-4c9a-a579-13405ab02c90 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:50.873069 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8ff95-a1c2-4a7a-b0a8-8e622677ad73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:50.873680 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8ff95-a1c2-4a7a-b0a8-8e622677ad73 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:50.873959 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8ff95-a1c2-4a7a-b0a8-8e622677ad73 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:50.874233 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8ff95-a1c2-4a7a-b0a8-8e622677ad73 None admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:50.874901 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8ff95-a1c2-4a7a-b0a8-8e622677ad73 None admin] RBAC: Adding query filter params (name=service) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:50.876680 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8ff95-a1c2-4a7a-b0a8-8e622677ad73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:50.878087 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8ff95-a1c2-4a7a-b0a8-8e622677ad73 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:50.878947 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08694a6a-cd76-497f-8a0d-9355cd9d5db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:50.879868 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08694a6a-cd76-497f-8a0d-9355cd9d5db9 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:50.880145 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08694a6a-cd76-497f-8a0d-9355cd9d5db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:50.880863 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08694a6a-cd76-497f-8a0d-9355cd9d5db9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:50.881595 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08694a6a-cd76-497f-8a0d-9355cd9d5db9 None admin] RBAC: Adding query filter params (role.id=b493ef4ee9d44f449e08251ae75a77d8, scope.project.id=02308892635844b088ecdcb5413a21a0, user.id=846251a661c04d3f9f195c4ac09b4189) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:50.881905 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08694a6a-cd76-497f-8a0d-9355cd9d5db9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:50.883369 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08694a6a-cd76-497f-8a0d-9355cd9d5db9 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:50.887234 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 245/490] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Oct 13 12:57:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:57:50.890771 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 246/491] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 13 12:57:50 2022] GET /identity/v3/role_assignments?user.id=846251a661c04d3f9f195c4ac09b4189&scope.project.id=02308892635844b088ecdcb5413a21a0&role.id=b493ef4ee9d44f449e08251ae75a77d8 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:50.896479 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8ff95-a1c2-4a7a-b0a8-8e622677ad73 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:50.899923 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:51.062524 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8ff95-a1c2-4a7a-b0a8-8e622677ad73 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:51.101864 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15bb7f3-322a-4639-800b-b51f644e8805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:51.102451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15bb7f3-322a-4639-800b-b51f644e8805 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:51.102732 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15bb7f3-322a-4639-800b-b51f644e8805 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:51.103005 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15bb7f3-322a-4639-800b-b51f644e8805 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:51.115345 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15bb7f3-322a-4639-800b-b51f644e8805 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:51.116830 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15bb7f3-322a-4639-800b-b51f644e8805 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:51.133088 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-a15bb7f3-322a-4639-800b-b51f644e8805 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:51.133088 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:51.133088 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:51.133088 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:51.133088 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:51.133088 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:51.133088 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:51.133088 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:51.133088 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:51.133088 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:51.133088 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:51.133088 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:51.133088 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:51.133088 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:51.133088 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:51.133088 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:51.134638 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:51.134638 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:51.134638 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:51.134638 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:51.134638 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:51.134638 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:51.134638 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:51.134638 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:51.134638 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:51.134638 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:51.134638 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:51.134638 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:51.134638 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:51.134638 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:51.134638 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:51.134638 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:51.134638 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:51.136885 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:51.136885 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:51.136885 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:51.136885 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 246/492] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:51.144354 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08694a6a-cd76-497f-8a0d-9355cd9d5db9 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:51.166036 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:51.263901 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08694a6a-cd76-497f-8a0d-9355cd9d5db9 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:51.300430 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed0e7f-6c0c-4013-aadf-e8b1080a6b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:51.301033 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed0e7f-6c0c-4013-aadf-e8b1080a6b6c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:51.301301 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed0e7f-6c0c-4013-aadf-e8b1080a6b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:51.301573 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed0e7f-6c0c-4013-aadf-e8b1080a6b6c None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:51.302212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ed0e7f-6c0c-4013-aadf-e8b1080a6b6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:51.302509 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ed0e7f-6c0c-4013-aadf-e8b1080a6b6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:51.303913 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ed0e7f-6c0c-4013-aadf-e8b1080a6b6c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:51.317181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 247/493] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:51.322763 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15bb7f3-322a-4639-800b-b51f644e8805 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:51.346787 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e68caf-d5cd-4cc0-b8ee-76f627c4783a None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:51.346787 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e68caf-d5cd-4cc0-b8ee-76f627c4783a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:51.346787 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e68caf-d5cd-4cc0-b8ee-76f627c4783a None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:51.346787 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 248/494] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 13 12:57:51.352119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d39ae4-1f06-41a7-8b28-9a2b8c19b755 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:51.352350 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d39ae4-1f06-41a7-8b28-9a2b8c19b755 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:51.352630 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d39ae4-1f06-41a7-8b28-9a2b8c19b755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:51.374711 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-e9d39ae4-1f06-41a7-8b28-9a2b8c19b755 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:51.375993 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e9d39ae4-1f06-41a7-8b28-9a2b8c19b755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:51.436713 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:51.499093 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15bb7f3-322a-4639-800b-b51f644e8805 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:51.532596 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 249/495] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 13 12:57:51.538143 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27b1e0-7d44-41b1-b497-843055ed8702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:51.538783 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27b1e0-7d44-41b1-b497-843055ed8702 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:51.539090 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27b1e0-7d44-41b1-b497-843055ed8702 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:51.539371 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27b1e0-7d44-41b1-b497-843055ed8702 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:51.548169 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27b1e0-7d44-41b1-b497-843055ed8702 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:51.549651 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27b1e0-7d44-41b1-b497-843055ed8702 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:51.564170 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-0b27b1e0-7d44-41b1-b497-843055ed8702 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 13 12:57:51.564170 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:51.564170 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:51.564170 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:51.564170 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:51.564170 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:51.564170 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:51.564170 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:51.564170 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:51.564170 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:51.564170 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:51.564170 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:51.564170 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 13 12:57:51.564170 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 13 12:57:51.564170 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 13 12:57:51.564170 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 13 12:57:51.565167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:51.565167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:51.565167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 13 12:57:51.565167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:57:51.565167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 13 12:57:51.565167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:57:51.565167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:51.565167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:51.565167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:51.565167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:51.565167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:51.565167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:51.565167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:51.565167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:51.565167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:51.565167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:51.565167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:51.565167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:51.566132 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:51.566132 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:51.566132 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 13 12:57:51.566132 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 13 12:57:51.566132 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 13 12:57:51.566132 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 13 12:57:51.566132 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 13 12:57:51.566132 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 13 12:57:51.566132 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 13 12:57:51.566132 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:51.566132 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 247/496] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 13 12:57:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:51.573092 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d39ae4-1f06-41a7-8b28-9a2b8c19b755 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:51.581212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:51.715047 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d39ae4-1f06-41a7-8b28-9a2b8c19b755 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:51.759769 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06112391-b39d-4434-8afc-881a0b1e9813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:51.760381 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06112391-b39d-4434-8afc-881a0b1e9813 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:51.760660 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06112391-b39d-4434-8afc-881a0b1e9813 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:51.760954 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06112391-b39d-4434-8afc-881a0b1e9813 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:51.761759 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06112391-b39d-4434-8afc-881a0b1e9813 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:51.762065 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06112391-b39d-4434-8afc-881a0b1e9813 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:51.763518 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06112391-b39d-4434-8afc-881a0b1e9813 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:51.784310 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 250/497] 10.222.0.23 () {56 vars in 1135 bytes} [Thu Oct 13 12:57:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:51.790087 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27b1e0-7d44-41b1-b497-843055ed8702 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:51.943229 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27b1e0-7d44-41b1-b497-843055ed8702 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:51.958200 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:51.979274 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76a91ac-5ce8-4ef3-b341-9c052bc8705e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:51.980352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a91ac-5ce8-4ef3-b341-9c052bc8705e None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:51.980634 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a91ac-5ce8-4ef3-b341-9c052bc8705e None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:51.980908 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a91ac-5ce8-4ef3-b341-9c052bc8705e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:52.003637 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06112391-b39d-4434-8afc-881a0b1e9813 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:52.085857 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76a91ac-5ce8-4ef3-b341-9c052bc8705e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:52.087423 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76a91ac-5ce8-4ef3-b341-9c052bc8705e None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:52.133097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:52.202452 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 248/498] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 13 12:57:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 13 12:57:52.208484 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76a91ac-5ce8-4ef3-b341-9c052bc8705e None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:52.226993 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06112391-b39d-4434-8afc-881a0b1e9813 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:52.251510 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:52.264701 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24cec7e-1ceb-4e9b-8892-4ff12dcb9048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:52.265698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cec7e-1ceb-4e9b-8892-4ff12dcb9048 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:52.266089 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cec7e-1ceb-4e9b-8892-4ff12dcb9048 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:52.266460 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cec7e-1ceb-4e9b-8892-4ff12dcb9048 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:52.278269 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24cec7e-1ceb-4e9b-8892-4ff12dcb9048 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:52.283402 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-b24cec7e-1ceb-4e9b-8892-4ff12dcb9048 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:57:52.283402 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:52.283402 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:52.283402 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:52.283402 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:52.283402 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:52.283402 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:52.283402 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:52.283402 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:52.283402 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:52.283402 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:52.283402 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:52.283402 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:52.283402 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:52.283402 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:52.283402 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:52.284382 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:52.284382 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:52.284382 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:52.284382 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:52.284382 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:52.284382 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:52.284382 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:52.284382 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:52.284382 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:52.284382 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:52.284382 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:52.284382 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:52.284382 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:52.284382 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:52.284382 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:52.284382 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:52.285202 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:52.285202 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:52.285202 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:52.285202 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:52.285202 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:52.285202 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:52.285202 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:52.285202 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:52.285202 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:52.285202 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:52.285202 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:57:52.285202 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:52.285826 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 251/499] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 13 12:57:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:52.292102 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24cec7e-1ceb-4e9b-8892-4ff12dcb9048 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:52.399734 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76a91ac-5ce8-4ef3-b341-9c052bc8705e None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:52.437541 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26501027-93c3-47a2-8085-bf04d7ee70c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:52.438191 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26501027-93c3-47a2-8085-bf04d7ee70c0 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:52.438477 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26501027-93c3-47a2-8085-bf04d7ee70c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:52.438755 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26501027-93c3-47a2-8085-bf04d7ee70c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:52.449892 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26501027-93c3-47a2-8085-bf04d7ee70c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:52.451448 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26501027-93c3-47a2-8085-bf04d7ee70c0 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:52.468451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-26501027-93c3-47a2-8085-bf04d7ee70c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:52.468451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:52.468451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:52.468451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:52.468451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:52.468451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:52.468451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:52.468451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:52.468451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:52.468451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:52.468451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:52.468451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:52.468451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:52.468451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:52.468451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:52.468451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:52.469365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:52.469365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:52.469365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:52.469365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:52.469365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:52.469365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:52.469365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:52.469365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:52.469365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:52.469365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:52.469365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:52.469365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:52.469365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:52.469365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:52.469365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:52.469365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:52.469365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:52.470725 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:52.470725 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:52.470725 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:52.470725 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 249/500] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:57:52.477199 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26501027-93c3-47a2-8085-bf04d7ee70c0 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:52.556963 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:52.573128 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24cec7e-1ceb-4e9b-8892-4ff12dcb9048 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:52.590133 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26501027-93c3-47a2-8085-bf04d7ee70c0 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:52.621723 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:52.627589 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2971974-c447-4cbe-ae6d-7c2e625319db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:52.628355 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2971974-c447-4cbe-ae6d-7c2e625319db None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:52.628642 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2971974-c447-4cbe-ae6d-7c2e625319db None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:52.628918 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2971974-c447-4cbe-ae6d-7c2e625319db None admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:52.629565 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2971974-c447-4cbe-ae6d-7c2e625319db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:52.629868 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2971974-c447-4cbe-ae6d-7c2e625319db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:52.631499 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2971974-c447-4cbe-ae6d-7c2e625319db None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:52.644162 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 250/501] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:57:52.650081 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2971974-c447-4cbe-ae6d-7c2e625319db None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:52.655759 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c6e75-f4d4-4e9b-9b48-2f1cdcb8ced0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:52.656380 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c6e75-f4d4-4e9b-9b48-2f1cdcb8ced0 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:52.656664 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c6e75-f4d4-4e9b-9b48-2f1cdcb8ced0 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:52.656941 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c6e75-f4d4-4e9b-9b48-2f1cdcb8ced0 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:52.657610 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73c6e75-f4d4-4e9b-9b48-2f1cdcb8ced0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:52.657910 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73c6e75-f4d4-4e9b-9b48-2f1cdcb8ced0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:52.659305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73c6e75-f4d4-4e9b-9b48-2f1cdcb8ced0 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:52.674928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 252/502] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:57:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:52.680552 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c6e75-f4d4-4e9b-9b48-2f1cdcb8ced0 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:52.758364 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2971974-c447-4cbe-ae6d-7c2e625319db None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:52.794276 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:52.801953 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f8738c-402f-431e-b992-50479c2b54ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:52.802650 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f8738c-402f-431e-b992-50479c2b54ad None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:52.803103 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f8738c-402f-431e-b992-50479c2b54ad None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:52.803429 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f8738c-402f-431e-b992-50479c2b54ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:52.822659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f8738c-402f-431e-b992-50479c2b54ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:52.824649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f8738c-402f-431e-b992-50479c2b54ad None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:52.850833 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-b7f8738c-402f-431e-b992-50479c2b54ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:57:52.850833 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:52.850833 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:52.850833 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:52.850833 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:52.850833 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:52.850833 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:52.850833 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:52.850833 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:52.850833 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:52.850833 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:52.850833 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:52.850833 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 13 12:57:52.850833 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 13 12:57:52.850833 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 13 12:57:52.851787 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 13 12:57:52.851787 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:52.851787 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:52.851787 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:52.851787 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:52.851787 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:52.851787 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:52.851787 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:52.851787 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:52.851787 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:52.851787 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:52.851787 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:52.851787 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:52.851787 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:52.851787 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:52.851787 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:52.851787 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:52.852694 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 13 12:57:52.852694 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 13 12:57:52.852694 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 13 12:57:52.852694 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 13 12:57:52.852694 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 13 12:57:52.852694 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 13 12:57:52.852694 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:57:52.852694 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:52.860053 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 251/503] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:57:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:57:52.870756 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c6e75-f4d4-4e9b-9b48-2f1cdcb8ced0 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:52.873550 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f8738c-402f-431e-b992-50479c2b54ad None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:52.920647 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602790c2-627f-483d-bffa-c7c7e71c2065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:52.921290 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602790c2-627f-483d-bffa-c7c7e71c2065 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:52.921569 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602790c2-627f-483d-bffa-c7c7e71c2065 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:52.921843 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602790c2-627f-483d-bffa-c7c7e71c2065 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:52.940228 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602790c2-627f-483d-bffa-c7c7e71c2065 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:52.942539 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602790c2-627f-483d-bffa-c7c7e71c2065 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:52.975961 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-602790c2-627f-483d-bffa-c7c7e71c2065 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:52.975961 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:52.975961 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:52.975961 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:52.975961 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:52.975961 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:52.975961 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:52.975961 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:52.975961 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:52.975961 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:52.975961 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:52.975961 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:52.975961 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:52.975961 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:52.975961 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:52.975961 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:52.977058 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:52.977058 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:52.977058 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:52.977058 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:52.977058 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:52.977058 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:52.977058 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:52.977058 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:52.977058 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:52.977058 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:52.977058 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:52.977058 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:52.977058 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:52.977058 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:52.977058 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:52.977058 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:52.977058 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:52.977939 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:52.977939 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:52.977939 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:52.977939 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 253/504] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:57:52.984703 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602790c2-627f-483d-bffa-c7c7e71c2065 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:53.028088 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:53.103944 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f8738c-402f-431e-b992-50479c2b54ad None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:53.122398 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602790c2-627f-483d-bffa-c7c7e71c2065 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:53.148134 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347167bf-bee3-4cea-b48c-66ff27d92381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:53.149101 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347167bf-bee3-4cea-b48c-66ff27d92381 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:53.149400 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347167bf-bee3-4cea-b48c-66ff27d92381 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:53.149950 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347167bf-bee3-4cea-b48c-66ff27d92381 None admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:53.150659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347167bf-bee3-4cea-b48c-66ff27d92381 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:53.151136 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347167bf-bee3-4cea-b48c-66ff27d92381 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:53.153541 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347167bf-bee3-4cea-b48c-66ff27d92381 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:53.159646 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a799749b-bd31-48bf-8236-f7ebc242ae07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:53.160331 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799749b-bd31-48bf-8236-f7ebc242ae07 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:53.160681 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799749b-bd31-48bf-8236-f7ebc242ae07 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:53.161025 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a799749b-bd31-48bf-8236-f7ebc242ae07 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:53.161733 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a799749b-bd31-48bf-8236-f7ebc242ae07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:53.162101 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a799749b-bd31-48bf-8236-f7ebc242ae07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:53.163554 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a799749b-bd31-48bf-8236-f7ebc242ae07 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:53.171166 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 252/505] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 13 12:57:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:53.172580 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7732f-98e2-49bf-8bd5-f120357f3a7f None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:53.172971 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7732f-98e2-49bf-8bd5-f120357f3a7f None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:53.173316 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7732f-98e2-49bf-8bd5-f120357f3a7f None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:53.176403 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 253/506] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:53.177518 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e7732f-98e2-49bf-8bd5-f120357f3a7f None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:53.178946 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 254/507] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:53.183669 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f6ccf-ded1-468f-a688-d3db6d54e0a8 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:53.184188 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f6ccf-ded1-468f-a688-d3db6d54e0a8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:53.184556 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f6ccf-ded1-468f-a688-d3db6d54e0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:53.205785 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8a4f6ccf-ded1-468f-a688-d3db6d54e0a8 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:53.207181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8a4f6ccf-ded1-468f-a688-d3db6d54e0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:53.230302 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:53.291208 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 255/508] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:53.291409 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e7732f-98e2-49bf-8bd5-f120357f3a7f None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:53.291496 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f6ccf-ded1-468f-a688-d3db6d54e0a8 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:53.325960 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39254a7-893c-4243-8b5f-ab2035f8c4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:53.326629 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:53.328152 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39254a7-893c-4243-8b5f-ab2035f8c4bf None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:53.328580 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39254a7-893c-4243-8b5f-ab2035f8c4bf None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:53.329228 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39254a7-893c-4243-8b5f-ab2035f8c4bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:53.330090 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39254a7-893c-4243-8b5f-ab2035f8c4bf None admin] RBAC: Adding query filter params (role.id=b493ef4ee9d44f449e08251ae75a77d8, scope.project.id=02308892635844b088ecdcb5413a21a0, user.id=526000341fd146db8c661f1fd14d4801) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:53.330669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39254a7-893c-4243-8b5f-ab2035f8c4bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:53.332884 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39254a7-893c-4243-8b5f-ab2035f8c4bf None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:53.340513 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 254/509] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 13 12:57:53 2022] GET /identity/v3/role_assignments?user.id=526000341fd146db8c661f1fd14d4801&scope.project.id=02308892635844b088ecdcb5413a21a0&role.id=b493ef4ee9d44f449e08251ae75a77d8 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:53.341974 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abfb85-1563-4c87-82e1-80ef038457b0 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:53.342392 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abfb85-1563-4c87-82e1-80ef038457b0 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:53.342793 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abfb85-1563-4c87-82e1-80ef038457b0 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:53.344260 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 255/510] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:53.356618 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a6b6b7-6e6c-4f53-b9b5-390800430b09 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:53.357110 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a6b6b7-6e6c-4f53-b9b5-390800430b09 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:53.357526 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a6b6b7-6e6c-4f53-b9b5-390800430b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:53.377996 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-e8a6b6b7-6e6c-4f53-b9b5-390800430b09 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:53.379533 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-e8a6b6b7-6e6c-4f53-b9b5-390800430b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:53.460383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 256/511] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 13 12:57:53.493211 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f6ccf-ded1-468f-a688-d3db6d54e0a8 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:53.531548 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28700860-b389-4f9a-9ef4-089c3d28484d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:53.532198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28700860-b389-4f9a-9ef4-089c3d28484d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:53.532480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28700860-b389-4f9a-9ef4-089c3d28484d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:53.532754 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28700860-b389-4f9a-9ef4-089c3d28484d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:53.541135 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28700860-b389-4f9a-9ef4-089c3d28484d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:53.542839 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86528b-47ae-468f-bf84-ce9e721658be None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:53.543175 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86528b-47ae-468f-bf84-ce9e721658be None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:53.543301 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28700860-b389-4f9a-9ef4-089c3d28484d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:53.543461 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86528b-47ae-468f-bf84-ce9e721658be None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:53.544948 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 257/512] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 13 12:57:53.556199 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-28700860-b389-4f9a-9ef4-089c3d28484d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 13 12:57:53.556199 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:53.556199 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:53.556199 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:53.556199 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:53.556199 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:53.556199 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:53.556199 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:53.556199 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:53.556199 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:53.556199 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:53.556199 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:53.556199 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 13 12:57:53.556199 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 13 12:57:53.556199 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 13 12:57:53.556199 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 13 12:57:53.557212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:53.557212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:53.557212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 13 12:57:53.557212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:57:53.557212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 13 12:57:53.557212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:57:53.557212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:53.557212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:53.557212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:53.557212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:53.557212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:53.557212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:53.557212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:53.557212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:53.557212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:53.557212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:53.557212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:53.557212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:53.559338 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:53.559338 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:53.559338 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 13 12:57:53.559338 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 13 12:57:53.559338 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 13 12:57:53.559338 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 13 12:57:53.559338 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 13 12:57:53.559338 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 13 12:57:53.559338 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 13 12:57:53.559338 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:53.559338 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 256/513] 10.222.0.23 () {56 vars in 1093 bytes} [Thu Oct 13 12:57:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:53.565143 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e6ed2-44de-45b2-a245-6d3224199e50 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:53.565143 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e6ed2-44de-45b2-a245-6d3224199e50 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:53.565143 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e6ed2-44de-45b2-a245-6d3224199e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:53.569281 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28700860-b389-4f9a-9ef4-089c3d28484d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:53.590919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-cc0e6ed2-44de-45b2-a245-6d3224199e50 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:53.592077 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-cc0e6ed2-44de-45b2-a245-6d3224199e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:53.634502 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:53.670332 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 258/514] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:53.671247 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e6ed2-44de-45b2-a245-6d3224199e50 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:53.719460 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:53.737261 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28700860-b389-4f9a-9ef4-089c3d28484d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:53.791849 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06460c6-3d3f-4dcc-af7e-4571899c6b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:53.792994 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06460c6-3d3f-4dcc-af7e-4571899c6b02 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:53.793275 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06460c6-3d3f-4dcc-af7e-4571899c6b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:53.793547 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06460c6-3d3f-4dcc-af7e-4571899c6b02 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:53.794278 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06460c6-3d3f-4dcc-af7e-4571899c6b02 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:53.794583 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06460c6-3d3f-4dcc-af7e-4571899c6b02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:53.796033 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06460c6-3d3f-4dcc-af7e-4571899c6b02 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:53.814140 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 257/515] 10.222.0.23 () {56 vars in 1133 bytes} [Thu Oct 13 12:57:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:53.814883 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06460c6-3d3f-4dcc-af7e-4571899c6b02 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:53.817747 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e6ed2-44de-45b2-a245-6d3224199e50 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:53.866202 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a47c58-de75-4870-aa89-741819aac0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:53.866839 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a47c58-de75-4870-aa89-741819aac0c2 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:53.867432 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a47c58-de75-4870-aa89-741819aac0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:53.867706 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a47c58-de75-4870-aa89-741819aac0c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:53.881533 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a47c58-de75-4870-aa89-741819aac0c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:53.882982 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a47c58-de75-4870-aa89-741819aac0c2 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:53.902489 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-44a47c58-de75-4870-aa89-741819aac0c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:53.902489 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:53.902489 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:53.902489 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:53.902489 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:53.902489 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:53.902489 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:53.902489 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:53.902489 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:53.902489 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:53.902489 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:53.902489 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:53.902489 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:53.902489 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:53.902489 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:53.902489 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:53.903484 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:53.903484 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:53.903484 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:53.903484 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:53.903484 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:53.903484 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:53.903484 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:53.903484 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:53.903484 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:53.903484 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:53.903484 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:53.903484 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:53.903484 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:53.903484 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:53.903484 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:53.903484 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:53.903484 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:53.904395 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:53.904395 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:53.904395 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:53.904395 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 259/516] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:53.905039 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a47c58-de75-4870-aa89-741819aac0c2 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:53.928858 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06460c6-3d3f-4dcc-af7e-4571899c6b02 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:53.942380 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:53.966963 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554534d-d120-4ccb-85cd-98f6eaba2fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:53.967620 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554534d-d120-4ccb-85cd-98f6eaba2fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:53.967894 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554534d-d120-4ccb-85cd-98f6eaba2fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:53.968159 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554534d-d120-4ccb-85cd-98f6eaba2fd5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:53.968736 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4554534d-d120-4ccb-85cd-98f6eaba2fd5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:53.970342 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4554534d-d120-4ccb-85cd-98f6eaba2fd5 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:53.982804 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-4554534d-d120-4ccb-85cd-98f6eaba2fd5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 13 12:57:53.982804 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:53.982804 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 13 12:57:53.982804 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 13 12:57:53.982804 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 13 12:57:53.982804 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 13 12:57:53.982804 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 13 12:57:53.982804 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:57:53.982804 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 13 12:57:53.982804 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:53.982804 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 12:57:53.982804 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:53.982804 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:53.982804 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:53.982804 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:53.982804 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:53.982804 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:53.984096 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:53.984096 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:53.984096 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:53.984096 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:53.984096 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:53.984096 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:53.984096 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 13 12:57:53.984096 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 13 12:57:53.984096 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 13 12:57:53.984096 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 13 12:57:53.984096 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:53.984096 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:53.984096 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:53.984096 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:53.984096 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:53.984096 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:53.989925 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:53.989925 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:53.989925 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:53.989925 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:53.989925 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:53.989925 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:53.989925 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:53.989925 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:53.989925 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:53.989925 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:53.989925 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 13 12:57:53.989925 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:57:53.989925 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 13 12:57:53.989925 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:53.989925 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 258/517] 10.222.0.23 () {56 vars in 1129 bytes} [Thu Oct 13 12:57:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:53.991198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554534d-d120-4ccb-85cd-98f6eaba2fd5 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:53.996085 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:54.036574 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a47c58-de75-4870-aa89-741819aac0c2 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:54.072974 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc8bc4-5bf4-4b26-bc2f-edcac11fff39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:54.073628 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc8bc4-5bf4-4b26-bc2f-edcac11fff39 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:54.073907 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc8bc4-5bf4-4b26-bc2f-edcac11fff39 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:54.074172 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc8bc4-5bf4-4b26-bc2f-edcac11fff39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:54.083949 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc8bc4-5bf4-4b26-bc2f-edcac11fff39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:54.085386 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc8bc4-5bf4-4b26-bc2f-edcac11fff39 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:54.093338 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554534d-d120-4ccb-85cd-98f6eaba2fd5 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:54.100068 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-8cdc8bc4-5bf4-4b26-bc2f-edcac11fff39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:54.100068 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:54.100068 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:54.100068 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:54.100068 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:54.100068 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:54.100068 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:54.100068 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:54.100068 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:54.100068 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:54.100068 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:54.100068 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:54.100068 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:54.100068 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:54.100068 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:54.100068 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:54.100983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:54.100983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:54.100983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:54.100983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:54.100983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:54.100983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:54.100983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:54.100983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:54.100983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:54.100983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:54.100983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:54.100983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:54.100983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:54.100983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:54.100983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:54.100983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:54.100983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:54.101895 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:54.101895 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:54.101895 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:54.101895 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 260/518] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:54.101895 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc8bc4-5bf4-4b26-bc2f-edcac11fff39 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:54.136593 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aae688-09a8-40a5-97de-320ff0874a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:54.137748 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aae688-09a8-40a5-97de-320ff0874a0e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:54.138211 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aae688-09a8-40a5-97de-320ff0874a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:54.138632 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aae688-09a8-40a5-97de-320ff0874a0e None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:54.139479 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aae688-09a8-40a5-97de-320ff0874a0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:54.139927 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aae688-09a8-40a5-97de-320ff0874a0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:54.141500 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aae688-09a8-40a5-97de-320ff0874a0e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:54.159825 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 259/519] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:54.159825 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aae688-09a8-40a5-97de-320ff0874a0e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:54.192059 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:54.220626 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc8bc4-5bf4-4b26-bc2f-edcac11fff39 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:54.228637 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:54.257802 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0628a36-3dd3-4df2-9527-8fea83fa0e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:54.258439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0628a36-3dd3-4df2-9527-8fea83fa0e5c None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:54.258882 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0628a36-3dd3-4df2-9527-8fea83fa0e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:54.259191 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0628a36-3dd3-4df2-9527-8fea83fa0e5c None admin] PATH_INFO: `/v3/services/network` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:54.260102 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0628a36-3dd3-4df2-9527-8fea83fa0e5c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:54.262476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0628a36-3dd3-4df2-9527-8fea83fa0e5c None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:54.265438 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aae688-09a8-40a5-97de-320ff0874a0e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:54.277479 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-f0628a36-3dd3-4df2-9527-8fea83fa0e5c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 13 12:57:54.277479 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:54.277479 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 13 12:57:54.277479 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 13 12:57:54.277479 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 13 12:57:54.277479 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 13 12:57:54.277479 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 13 12:57:54.277479 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:57:54.277479 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 13 12:57:54.277479 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:54.277479 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 12:57:54.277479 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:54.277479 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:54.277479 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:54.277479 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:54.277479 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:54.277479 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:54.278468 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:54.278468 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:54.278468 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:54.278468 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:54.278468 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:54.278468 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:54.278468 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 13 12:57:54.278468 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 13 12:57:54.278468 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 13 12:57:54.278468 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 13 12:57:54.278468 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:54.278468 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:54.278468 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:54.278468 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:54.278468 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:54.278468 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:54.279948 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:54.279948 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:54.279948 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:54.279948 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:54.279948 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:54.279948 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:54.279948 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:54.279948 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:54.279948 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:54.279948 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:54.279948 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 13 12:57:54.279948 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:57:54.279948 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 13 12:57:54.279948 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:54.281071 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 261/520] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:57:54 2022] GET /identity/v3/services/network => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:54.281977 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0628a36-3dd3-4df2-9527-8fea83fa0e5c None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:54.304459 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a128e64-eb0f-4da1-9384-e0edfa027829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:54.305336 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a128e64-eb0f-4da1-9384-e0edfa027829 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:54.305790 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a128e64-eb0f-4da1-9384-e0edfa027829 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:54.306235 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a128e64-eb0f-4da1-9384-e0edfa027829 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:54.307094 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a128e64-eb0f-4da1-9384-e0edfa027829 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:54.307468 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a128e64-eb0f-4da1-9384-e0edfa027829 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:54.308910 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a128e64-eb0f-4da1-9384-e0edfa027829 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:54.318136 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 260/521] 10.222.0.23 () {56 vars in 1124 bytes} [Thu Oct 13 12:57:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:54.318136 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a128e64-eb0f-4da1-9384-e0edfa027829 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:54.388174 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0628a36-3dd3-4df2-9527-8fea83fa0e5c None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:54.422667 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a128e64-eb0f-4da1-9384-e0edfa027829 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:54.423122 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c96e8-603b-4166-b61a-6bcccd7deec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:54.423735 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c96e8-603b-4166-b61a-6bcccd7deec3 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:54.424017 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c96e8-603b-4166-b61a-6bcccd7deec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:54.424298 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c96e8-603b-4166-b61a-6bcccd7deec3 None admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:54.424933 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9c96e8-603b-4166-b61a-6bcccd7deec3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:54.425256 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9c96e8-603b-4166-b61a-6bcccd7deec3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:54.426615 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9c96e8-603b-4166-b61a-6bcccd7deec3 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:54.433077 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:54.443904 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 262/522] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:54.443904 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c96e8-603b-4166-b61a-6bcccd7deec3 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:54.460130 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:54.466673 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699750f-4107-42a5-bf25-8fbe100f71c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:54.467424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699750f-4107-42a5-bf25-8fbe100f71c1 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:54.467808 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699750f-4107-42a5-bf25-8fbe100f71c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:54.468179 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699750f-4107-42a5-bf25-8fbe100f71c1 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:54.468942 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3699750f-4107-42a5-bf25-8fbe100f71c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:54.470473 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3699750f-4107-42a5-bf25-8fbe100f71c1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:54.499128 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 261/523] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 13 12:57:54 2022] POST /identity/v3/users => generated 254 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:57:54.499128 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699750f-4107-42a5-bf25-8fbe100f71c1 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:54.551380 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c96e8-603b-4166-b61a-6bcccd7deec3 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:54.598832 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a767794-9b28-4346-aa55-c8b702062c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:54.602305 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a767794-9b28-4346-aa55-c8b702062c05 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:54.603593 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a767794-9b28-4346-aa55-c8b702062c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:54.604152 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a767794-9b28-4346-aa55-c8b702062c05 None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:54.605399 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a767794-9b28-4346-aa55-c8b702062c05 None admin] RBAC: Adding query filter params (name=network) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:54.605914 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a767794-9b28-4346-aa55-c8b702062c05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:54.608826 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a767794-9b28-4346-aa55-c8b702062c05 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:54.615258 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 263/524] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Oct 13 12:57:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:54.616244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a767794-9b28-4346-aa55-c8b702062c05 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:54.649745 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699750f-4107-42a5-bf25-8fbe100f71c1 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:54.688845 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:54.708997 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab90b3-5de7-46e1-b578-116978f85e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:54.709651 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab90b3-5de7-46e1-b578-116978f85e17 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:54.709964 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab90b3-5de7-46e1-b578-116978f85e17 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:54.710238 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab90b3-5de7-46e1-b578-116978f85e17 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:54.710879 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab90b3-5de7-46e1-b578-116978f85e17 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:54.711330 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab90b3-5de7-46e1-b578-116978f85e17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:54.712592 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab90b3-5de7-46e1-b578-116978f85e17 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:54.720347 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 262/525] 10.222.0.23 () {56 vars in 1124 bytes} [Thu Oct 13 12:57:54 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:54.721846 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab90b3-5de7-46e1-b578-116978f85e17 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:54.747315 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a767794-9b28-4346-aa55-c8b702062c05 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:54.796924 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91215e2b-19d2-45c9-a605-0a4928c581cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:54.798214 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:54.799354 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91215e2b-19d2-45c9-a605-0a4928c581cd None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:54.800123 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91215e2b-19d2-45c9-a605-0a4928c581cd None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:54.801423 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91215e2b-19d2-45c9-a605-0a4928c581cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:54.813869 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91215e2b-19d2-45c9-a605-0a4928c581cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:54.815659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91215e2b-19d2-45c9-a605-0a4928c581cd None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:54.833939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-91215e2b-19d2-45c9-a605-0a4928c581cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:57:54.833939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:54.833939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:54.833939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:54.833939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:54.833939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:54.833939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:54.833939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:54.833939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:54.833939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:54.833939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:54.833939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:54.833939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 13 12:57:54.833939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 13 12:57:54.833939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 13 12:57:54.835538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 13 12:57:54.835538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:54.835538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:54.835538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:54.835538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:54.835538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:54.835538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:54.835538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:54.835538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:54.835538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:54.835538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:54.835538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:54.835538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:54.835538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:54.835538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:54.835538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:54.835538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:54.836603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 13 12:57:54.836603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 13 12:57:54.836603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 13 12:57:54.836603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 13 12:57:54.836603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 13 12:57:54.836603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 13 12:57:54.836603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:57:54.836603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:54.839744 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 264/526] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:57:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:54.846847 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91215e2b-19d2-45c9-a605-0a4928c581cd None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:54.881346 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab90b3-5de7-46e1-b578-116978f85e17 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:54.960711 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b98fe-7ef2-49c9-93e2-3ad1e1a9b34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:54.962182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b98fe-7ef2-49c9-93e2-3ad1e1a9b34b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:54.962668 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b98fe-7ef2-49c9-93e2-3ad1e1a9b34b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:54.963226 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b98fe-7ef2-49c9-93e2-3ad1e1a9b34b None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:54.964255 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b98fe-7ef2-49c9-93e2-3ad1e1a9b34b None admin] RBAC: Adding query filter params (type=network) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:54.964853 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b98fe-7ef2-49c9-93e2-3ad1e1a9b34b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:54.966437 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b98fe-7ef2-49c9-93e2-3ad1e1a9b34b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:54.982376 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 263/527] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Oct 13 12:57:54 2022] GET /identity/v3/services?type=network => generated 124 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:55.013049 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91215e2b-19d2-45c9-a605-0a4928c581cd None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:55.048849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6161da3-1a64-4e72-9d94-f6ece23e3af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:55.049482 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6161da3-1a64-4e72-9d94-f6ece23e3af8 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:55.049771 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6161da3-1a64-4e72-9d94-f6ece23e3af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:55.050049 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6161da3-1a64-4e72-9d94-f6ece23e3af8 None admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:55.050931 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6161da3-1a64-4e72-9d94-f6ece23e3af8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:55.051160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6161da3-1a64-4e72-9d94-f6ece23e3af8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:55.052576 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6161da3-1a64-4e72-9d94-f6ece23e3af8 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:55.061691 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:55.067034 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 265/528] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 13 12:57:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:55.073950 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b98fe-7ef2-49c9-93e2-3ad1e1a9b34b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:55.205476 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b98fe-7ef2-49c9-93e2-3ad1e1a9b34b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:55.233566 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:55.263953 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5222c7-c193-47d0-9c05-93e3cca1bd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:55.264964 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5222c7-c193-47d0-9c05-93e3cca1bd51 None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:55.265421 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5222c7-c193-47d0-9c05-93e3cca1bd51 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:55.265868 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5222c7-c193-47d0-9c05-93e3cca1bd51 None admin] PATH_INFO: `/v3/projects/02308892635844b088ecdcb5413a21a0/users/12700d985a724595af630c81e63f86d7/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:55.267968 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5222c7-c193-47d0-9c05-93e3cca1bd51 None admin] RBAC: Authorizing `identity:create_grant(project_id=02308892635844b088ecdcb5413a21a0, user_id=12700d985a724595af630c81e63f86d7, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:55.269576 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5222c7-c193-47d0-9c05-93e3cca1bd51 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:55.294679 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 264/529] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 13 12:57:55 2022] PUT /identity/v3/projects/02308892635844b088ecdcb5413a21a0/users/12700d985a724595af630c81e63f86d7/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 12:57:55.740327 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5654960-575f-499a-8384-00b09f20b9f0 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:55.740635 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5654960-575f-499a-8384-00b09f20b9f0 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:55.741099 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5654960-575f-499a-8384-00b09f20b9f0 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:55.742808 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 266/530] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:55.748823 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88086a5c-9797-48d0-8725-a7b2529cb115 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:55.749198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88086a5c-9797-48d0-8725-a7b2529cb115 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:55.749549 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88086a5c-9797-48d0-8725-a7b2529cb115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:55.769186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-88086a5c-9797-48d0-8725-a7b2529cb115 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:55.770504 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-88086a5c-9797-48d0-8725-a7b2529cb115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:55.840297 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 265/531] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:56.154637 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5654960-575f-499a-8384-00b09f20b9f0 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:56.285518 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5654960-575f-499a-8384-00b09f20b9f0 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:56.320599 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4cbec0-42ee-481d-99d9-ae512ed6c243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:56.321231 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4cbec0-42ee-481d-99d9-ae512ed6c243 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:56.321511 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4cbec0-42ee-481d-99d9-ae512ed6c243 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:56.321785 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4cbec0-42ee-481d-99d9-ae512ed6c243 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:56.332529 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4cbec0-42ee-481d-99d9-ae512ed6c243 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:56.337341 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-ab4cbec0-42ee-481d-99d9-ae512ed6c243 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:57:56.337341 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:56.337341 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:56.337341 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:56.337341 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:56.337341 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:56.337341 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:56.337341 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:56.337341 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:56.337341 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:56.337341 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:56.337341 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:56.337341 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:56.337341 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:56.337341 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:56.337341 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:56.338244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:56.338244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:56.338244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:56.338244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:56.338244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:56.338244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:56.338244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:56.338244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:56.338244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:56.338244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:56.338244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:56.338244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:56.338244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:56.338244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:56.338244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:56.338244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:56.339649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:56.339649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:56.339649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:56.339649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:56.339649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:56.339649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:56.339649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:56.339649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:56.339649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:56.339649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:56.339649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:57:56.339649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:56.343165 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 267/532] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 13 12:57:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:56.345286 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88086a5c-9797-48d0-8725-a7b2529cb115 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:56.461857 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88086a5c-9797-48d0-8725-a7b2529cb115 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:56.470265 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:56.506516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6176f6-7163-4a7a-89c3-f01d2ec52120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:56.507167 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6176f6-7163-4a7a-89c3-f01d2ec52120 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:56.507451 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6176f6-7163-4a7a-89c3-f01d2ec52120 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:56.507723 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6176f6-7163-4a7a-89c3-f01d2ec52120 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:56.508391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6176f6-7163-4a7a-89c3-f01d2ec52120 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:56.508694 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6176f6-7163-4a7a-89c3-f01d2ec52120 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:56.510060 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6176f6-7163-4a7a-89c3-f01d2ec52120 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:56.523445 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 266/533] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:57:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:56.525192 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4cbec0-42ee-481d-99d9-ae512ed6c243 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:56.578767 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:56.649409 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4cbec0-42ee-481d-99d9-ae512ed6c243 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:56.695583 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df35bb-df23-43fe-825e-c622329c9c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:56.696569 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df35bb-df23-43fe-825e-c622329c9c9f None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:56.697054 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df35bb-df23-43fe-825e-c622329c9c9f None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:56.703077 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df35bb-df23-43fe-825e-c622329c9c9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:56.714464 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df35bb-df23-43fe-825e-c622329c9c9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:56.716197 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df35bb-df23-43fe-825e-c622329c9c9f None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:56.735340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-f8df35bb-df23-43fe-825e-c622329c9c9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:56.735340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:56.735340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:56.735340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:56.735340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:56.735340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:56.735340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:56.735340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:56.735340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:56.735340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:56.735340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:56.735340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:56.735340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:56.735340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:56.735340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:56.735340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:56.736281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:56.736281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:56.736281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:56.736281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:56.736281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:56.736281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:56.736281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:56.736281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:56.736281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:56.736281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:56.736281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:56.736281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:56.736281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:56.736281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:56.736281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:56.736281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:56.736281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:56.737155 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:56.737155 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:56.737155 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:56.738878 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 268/534] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:56.745333 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6176f6-7163-4a7a-89c3-f01d2ec52120 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:56.915516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6176f6-7163-4a7a-89c3-f01d2ec52120 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:56.950732 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3cbbc-50f5-4a96-be73-329d5bbf26a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:56.951388 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3cbbc-50f5-4a96-be73-329d5bbf26a0 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:56.951670 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3cbbc-50f5-4a96-be73-329d5bbf26a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:56.951950 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3cbbc-50f5-4a96-be73-329d5bbf26a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:56.952587 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3cbbc-50f5-4a96-be73-329d5bbf26a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:56.952894 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3cbbc-50f5-4a96-be73-329d5bbf26a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:56.954252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3cbbc-50f5-4a96-be73-329d5bbf26a0 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:56.967447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 267/535] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:56.980209 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df35bb-df23-43fe-825e-c622329c9c9f None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:57.213916 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:57.233763 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df35bb-df23-43fe-825e-c622329c9c9f None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:57.319476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b11fd-cabf-4769-bf56-70cce9b17043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:57.320124 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b11fd-cabf-4769-bf56-70cce9b17043 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:57.320410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b11fd-cabf-4769-bf56-70cce9b17043 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:57.320678 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b11fd-cabf-4769-bf56-70cce9b17043 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:57.333920 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4b11fd-cabf-4769-bf56-70cce9b17043 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:57.335462 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4b11fd-cabf-4769-bf56-70cce9b17043 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:57.352673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-db4b11fd-cabf-4769-bf56-70cce9b17043 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 13 12:57:57.352673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:57.352673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:57.352673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:57.352673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:57.352673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:57.352673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:57.352673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:57.352673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:57.352673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:57.352673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:57.352673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:57.352673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 13 12:57:57.352673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 13 12:57:57.352673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 13 12:57:57.352673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 13 12:57:57.353616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:57.353616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:57.353616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 13 12:57:57.353616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:57:57.353616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 13 12:57:57.353616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:57:57.353616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:57.353616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:57.353616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:57.353616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:57.353616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:57.353616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:57.353616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:57.353616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:57.353616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:57.353616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:57.353616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:57.353616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:57.355100 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:57.355100 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:57.355100 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 13 12:57:57.355100 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 13 12:57:57.355100 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 13 12:57:57.355100 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 13 12:57:57.355100 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 13 12:57:57.355100 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 13 12:57:57.355100 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 13 12:57:57.355100 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:57.355100 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 269/536] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 13 12:57:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:57.360182 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b11fd-cabf-4769-bf56-70cce9b17043 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:57.360712 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2e2e4-a8e4-46ca-908d-2c20c5736edc None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:57.361232 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2e2e4-a8e4-46ca-908d-2c20c5736edc None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:57.361397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2e2e4-a8e4-46ca-908d-2c20c5736edc None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:57.362837 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 268/537] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:57.376511 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361f88d-7c97-4a1f-8399-924e140435d1 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:57.376793 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361f88d-7c97-4a1f-8399-924e140435d1 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:57.377045 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361f88d-7c97-4a1f-8399-924e140435d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:57.423144 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-e361f88d-7c97-4a1f-8399-924e140435d1 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:57.424316 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e361f88d-7c97-4a1f-8399-924e140435d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:57.475739 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:57.534164 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 269/538] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:57.535044 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b11fd-cabf-4769-bf56-70cce9b17043 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:57.535547 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e677f76-87a2-47eb-afee-eb64e047fd46 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:57.535922 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e677f76-87a2-47eb-afee-eb64e047fd46 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:57.536161 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e677f76-87a2-47eb-afee-eb64e047fd46 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:57.538478 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 270/539] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:57.544942 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ea2ce-2e71-4d8e-ac81-e962203b96e6 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:57.545227 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ea2ce-2e71-4d8e-ac81-e962203b96e6 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:57.545500 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ea2ce-2e71-4d8e-ac81-e962203b96e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:57.567375 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b00ea2ce-2e71-4d8e-ac81-e962203b96e6 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:57.568759 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b00ea2ce-2e71-4d8e-ac81-e962203b96e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:57.611312 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c5a25-f55d-4d7f-92c3-b368e6a02cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:57.611921 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c5a25-f55d-4d7f-92c3-b368e6a02cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:57.612194 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c5a25-f55d-4d7f-92c3-b368e6a02cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:57.612465 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c5a25-f55d-4d7f-92c3-b368e6a02cd8 None admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:57.613201 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c5a25-f55d-4d7f-92c3-b368e6a02cd8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:57.613507 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c5a25-f55d-4d7f-92c3-b368e6a02cd8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:57.614946 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c5a25-f55d-4d7f-92c3-b368e6a02cd8 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:57.653527 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 270/540] 10.222.0.23 () {56 vars in 1135 bytes} [Thu Oct 13 12:57:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:57.658239 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c5a25-f55d-4d7f-92c3-b368e6a02cd8 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:57.681006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:57.688449 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 271/541] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 13 12:57:57.790757 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c5a25-f55d-4d7f-92c3-b368e6a02cd8 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:57.825048 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ea2ce-2e71-4d8e-ac81-e962203b96e6 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:57.837763 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eefb0e-32a5-4803-9508-b71a0ae37f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:57.838407 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eefb0e-32a5-4803-9508-b71a0ae37f16 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:57.839216 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eefb0e-32a5-4803-9508-b71a0ae37f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:57.839507 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eefb0e-32a5-4803-9508-b71a0ae37f16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:57.863479 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eefb0e-32a5-4803-9508-b71a0ae37f16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:57.864983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eefb0e-32a5-4803-9508-b71a0ae37f16 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:57.899006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-66eefb0e-32a5-4803-9508-b71a0ae37f16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:57.899006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:57.899006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:57.899006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:57.899006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:57.899006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:57.899006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:57.899006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:57.899006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:57.899006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:57.899006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:57.899006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:57.899006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:57.899006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:57.899006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:57.899006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:57.900021 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:57.900021 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:57.900021 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:57.900021 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:57.900021 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:57.900021 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:57.900021 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:57.900021 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:57.900021 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:57.900021 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:57.900021 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:57.900021 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:57.900021 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:57.900021 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:57.900021 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:57.900021 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:57.900021 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:57.900874 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:57.900874 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:57.900874 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:57.902832 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 271/542] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:57.910159 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eefb0e-32a5-4803-9508-b71a0ae37f16 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:57.990518 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:58.004577 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:58.005174 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ea2ce-2e71-4d8e-ac81-e962203b96e6 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:58.040595 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc778b22-0fa2-4eb7-9413-1e5793ebe950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:58.041277 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc778b22-0fa2-4eb7-9413-1e5793ebe950 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:58.041501 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc778b22-0fa2-4eb7-9413-1e5793ebe950 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:58.041774 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc778b22-0fa2-4eb7-9413-1e5793ebe950 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:58.045589 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eefb0e-32a5-4803-9508-b71a0ae37f16 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:58.051837 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc778b22-0fa2-4eb7-9413-1e5793ebe950 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:58.056666 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-dc778b22-0fa2-4eb7-9413-1e5793ebe950 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 13 12:57:58.056666 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:58.056666 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:58.056666 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:58.056666 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:58.056666 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:58.056666 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:58.056666 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:58.056666 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:58.056666 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:58.056666 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:58.056666 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:58.056666 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:58.056666 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:58.056666 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:58.056666 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:58.057620 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:58.057620 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:58.057620 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:58.057620 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:58.057620 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:58.057620 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:58.057620 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:58.057620 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:58.057620 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:58.057620 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:58.057620 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:58.057620 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:58.057620 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:58.057620 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:58.057620 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:58.057620 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:58.058480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:58.058480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:58.058480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:58.058480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:58.058480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:58.058480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:58.058480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:58.058480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:58.058480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:58.058480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:58.058480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 13 12:57:58.058480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:58.058480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 272/543] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Oct 13 12:57:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:58.059705 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3551f-a328-4a16-ab02-2e074f63f537 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:58.059705 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3551f-a328-4a16-ab02-2e074f63f537 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:58.060017 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3551f-a328-4a16-ab02-2e074f63f537 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:58.065415 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 273/544] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:58.066201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3551f-a328-4a16-ab02-2e074f63f537 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:58.081740 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9a75d-5dbb-4df0-b8b0-8ff310fe0085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:58.082353 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9a75d-5dbb-4df0-b8b0-8ff310fe0085 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:58.082635 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9a75d-5dbb-4df0-b8b0-8ff310fe0085 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:58.082907 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9a75d-5dbb-4df0-b8b0-8ff310fe0085 None admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:58.083577 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9a75d-5dbb-4df0-b8b0-8ff310fe0085 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:58.083881 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9a75d-5dbb-4df0-b8b0-8ff310fe0085 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:58.085257 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9a75d-5dbb-4df0-b8b0-8ff310fe0085 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:58.097576 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 272/545] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:58.097576 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9a75d-5dbb-4df0-b8b0-8ff310fe0085 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:58.180217 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3551f-a328-4a16-ab02-2e074f63f537 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:58.205373 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9a75d-5dbb-4df0-b8b0-8ff310fe0085 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:58.217535 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839062f3-a4dd-4ff3-beea-9d1ae2682a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:58.218153 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839062f3-a4dd-4ff3-beea-9d1ae2682a37 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:58.218548 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839062f3-a4dd-4ff3-beea-9d1ae2682a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:58.218829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839062f3-a4dd-4ff3-beea-9d1ae2682a37 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:58.219651 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839062f3-a4dd-4ff3-beea-9d1ae2682a37 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:58.221056 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839062f3-a4dd-4ff3-beea-9d1ae2682a37 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:58.229800 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:58.251095 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 274/546] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Oct 13 12:57:58 2022] POST /identity/v3/services => generated 251 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:57:58.251095 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf424f63-3684-4c85-9aed-a8fd76ea10d4 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:58.251095 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf424f63-3684-4c85-9aed-a8fd76ea10d4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:58.251095 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf424f63-3684-4c85-9aed-a8fd76ea10d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:58.259975 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6335aa5-e6a3-4004-a997-a12d09bd1772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:58.261593 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6335aa5-e6a3-4004-a997-a12d09bd1772 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:58.262098 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6335aa5-e6a3-4004-a997-a12d09bd1772 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:58.262594 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6335aa5-e6a3-4004-a997-a12d09bd1772 None admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:58.263650 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6335aa5-e6a3-4004-a997-a12d09bd1772 None admin] RBAC: Adding query filter params (name=service) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:58.264239 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6335aa5-e6a3-4004-a997-a12d09bd1772 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:58.265862 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6335aa5-e6a3-4004-a997-a12d09bd1772 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:58.269346 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-cf424f63-3684-4c85-9aed-a8fd76ea10d4 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:58.270650 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-cf424f63-3684-4c85-9aed-a8fd76ea10d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:58.274266 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:58.279424 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 273/547] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Oct 13 12:57:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:58.281273 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6335aa5-e6a3-4004-a997-a12d09bd1772 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:58.355084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 275/548] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:58 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:58.355084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf424f63-3684-4c85-9aed-a8fd76ea10d4 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:58.435466 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6335aa5-e6a3-4004-a997-a12d09bd1772 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:58.473531 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf424f63-3684-4c85-9aed-a8fd76ea10d4 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:58.478617 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97732ae-ffcb-4308-a40e-3d7f6e503ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:58.479322 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97732ae-ffcb-4308-a40e-3d7f6e503ba3 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:58.479609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97732ae-ffcb-4308-a40e-3d7f6e503ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:58.479880 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97732ae-ffcb-4308-a40e-3d7f6e503ba3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:58.487969 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97732ae-ffcb-4308-a40e-3d7f6e503ba3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:58.489431 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97732ae-ffcb-4308-a40e-3d7f6e503ba3 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:58.501962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-d97732ae-ffcb-4308-a40e-3d7f6e503ba3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:57:58.501962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:58.501962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:58.501962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:58.501962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:58.501962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:58.501962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:58.501962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:58.501962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:58.501962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:58.501962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:58.501962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:58.501962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 13 12:57:58.501962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 13 12:57:58.501962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 13 12:57:58.502867 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 13 12:57:58.502867 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:58.502867 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:58.502867 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:58.502867 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:58.502867 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:58.502867 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:58.502867 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:58.502867 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:58.502867 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:58.502867 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:58.502867 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:58.502867 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:58.502867 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:58.502867 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:58.502867 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:58.502867 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:58.503777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 13 12:57:58.503777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 13 12:57:58.503777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 13 12:57:58.503777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 13 12:57:58.503777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 13 12:57:58.503777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 13 12:57:58.503777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:57:58.503777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:58.503777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 274/549] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:57:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:58.505422 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddf002-971c-45e2-b9fe-7c89c1179527 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:58.505867 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddf002-971c-45e2-b9fe-7c89c1179527 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:58.506136 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddf002-971c-45e2-b9fe-7c89c1179527 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:58.512348 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 275/550] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:57:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:57:58.513163 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ddf002-971c-45e2-b9fe-7c89c1179527 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:58.520587 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60f128-35ff-457f-80f8-5d7444bdebfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:58.521448 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60f128-35ff-457f-80f8-5d7444bdebfc None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:58.521928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60f128-35ff-457f-80f8-5d7444bdebfc None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:58.522800 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60f128-35ff-457f-80f8-5d7444bdebfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:58.530918 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:58.534973 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60f128-35ff-457f-80f8-5d7444bdebfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:58.537237 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60f128-35ff-457f-80f8-5d7444bdebfc None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:58.554682 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-2d60f128-35ff-457f-80f8-5d7444bdebfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:58.554682 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:58.554682 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:58.554682 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:58.554682 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:58.554682 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:58.554682 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:58.554682 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:58.554682 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:58.554682 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:58.554682 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:58.554682 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:58.554682 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:58.554682 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:58.554682 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:58.554682 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:58.555745 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:58.555745 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:58.555745 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:58.555745 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:58.555745 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:58.555745 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:58.555745 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:58.555745 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:58.555745 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:58.555745 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:58.555745 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:58.555745 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:58.555745 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:58.555745 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:58.555745 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:58.555745 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:58.555745 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:58.556657 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:58.556657 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:58.556657 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:58.560194 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 276/551] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:58.563471 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a3809-d983-4c89-8b4f-7c37845a717c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:58.567424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a3809-d983-4c89-8b4f-7c37845a717c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:58.567807 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a3809-d983-4c89-8b4f-7c37845a717c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:58.570933 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:58.586806 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c93a3809-d983-4c89-8b4f-7c37845a717c None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:57:58.588014 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c93a3809-d983-4c89-8b4f-7c37845a717c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:57:58.633616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ddf002-971c-45e2-b9fe-7c89c1179527 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:58.655464 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 277/552] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:57:58 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 13 12:57:58.656314 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a3809-d983-4c89-8b4f-7c37845a717c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:58.681694 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6120a15-109b-4408-b4cf-4242a6dcb82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:58.682310 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6120a15-109b-4408-b4cf-4242a6dcb82f None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:58.683165 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6120a15-109b-4408-b4cf-4242a6dcb82f None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:58.683440 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6120a15-109b-4408-b4cf-4242a6dcb82f None admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:58.684489 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6120a15-109b-4408-b4cf-4242a6dcb82f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:58.684598 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6120a15-109b-4408-b4cf-4242a6dcb82f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:58.685968 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6120a15-109b-4408-b4cf-4242a6dcb82f None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:58.700107 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 276/553] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 13 12:57:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:58.700917 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6120a15-109b-4408-b4cf-4242a6dcb82f None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:58.784655 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a3809-d983-4c89-8b4f-7c37845a717c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:58.812377 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6120a15-109b-4408-b4cf-4242a6dcb82f None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:58.813795 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:58.814374 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:58.833532 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9946f0e-c691-4161-881f-1c6f7ee29ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:58.834845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9946f0e-c691-4161-881f-1c6f7ee29ec9 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:58.835348 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9946f0e-c691-4161-881f-1c6f7ee29ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:58.835812 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9946f0e-c691-4161-881f-1c6f7ee29ec9 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:58.836700 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9946f0e-c691-4161-881f-1c6f7ee29ec9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:58.837686 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9946f0e-c691-4161-881f-1c6f7ee29ec9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:58.839859 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9946f0e-c691-4161-881f-1c6f7ee29ec9 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:58.851460 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45c204-0634-4daf-a505-915a35f65e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:58.852160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45c204-0634-4daf-a505-915a35f65e57 None admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:58.852358 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45c204-0634-4daf-a505-915a35f65e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:58.852620 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45c204-0634-4daf-a505-915a35f65e57 None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:58.853273 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac45c204-0634-4daf-a505-915a35f65e57 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:58.855216 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac45c204-0634-4daf-a505-915a35f65e57 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:58.855770 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 278/554] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:58.856802 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9946f0e-c691-4161-881f-1c6f7ee29ec9 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:58.873992 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 277/555] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Oct 13 12:57:58 2022] POST /identity/v3/services => generated 240 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:57:58.874987 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45c204-0634-4daf-a505-915a35f65e57 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:58.978605 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9946f0e-c691-4161-881f-1c6f7ee29ec9 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:59.001814 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45c204-0634-4daf-a505-915a35f65e57 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:59.022427 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb789aeb-ec15-41cf-ab44-d4fe836a7773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:59.023510 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb789aeb-ec15-41cf-ab44-d4fe836a7773 None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:59.023996 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb789aeb-ec15-41cf-ab44-d4fe836a7773 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:59.024473 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb789aeb-ec15-41cf-ab44-d4fe836a7773 None admin] PATH_INFO: `/v3/projects/02308892635844b088ecdcb5413a21a0/users/526000341fd146db8c661f1fd14d4801/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:59.026872 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb789aeb-ec15-41cf-ab44-d4fe836a7773 None admin] RBAC: Authorizing `identity:create_grant(project_id=02308892635844b088ecdcb5413a21a0, user_id=526000341fd146db8c661f1fd14d4801, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:59.028663 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb789aeb-ec15-41cf-ab44-d4fe836a7773 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:59.044645 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad27c3-60db-408d-a43e-c114903a592e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:59.045299 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad27c3-60db-408d-a43e-c114903a592e None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:59.045579 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad27c3-60db-408d-a43e-c114903a592e None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:59.045859 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad27c3-60db-408d-a43e-c114903a592e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:59.047797 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 279/556] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 13 12:57:58 2022] PUT /identity/v3/projects/02308892635844b088ecdcb5413a21a0/users/526000341fd146db8c661f1fd14d4801/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 12:57:59.049926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb789aeb-ec15-41cf-ab44-d4fe836a7773 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:59.060189 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad27c3-60db-408d-a43e-c114903a592e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:59.060189 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad27c3-60db-408d-a43e-c114903a592e None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:59.073451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-44ad27c3-60db-408d-a43e-c114903a592e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 13 12:57:59.073451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:59.073451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:59.073451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:59.073451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:59.073451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:59.073451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:59.073451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:59.073451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:59.073451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:59.073451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:59.073451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:59.073451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 13 12:57:59.073451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 13 12:57:59.073451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 13 12:57:59.073451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 13 12:57:59.074465 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:59.074465 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:59.074465 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 13 12:57:59.074465 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:57:59.074465 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 13 12:57:59.074465 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:57:59.074465 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:59.074465 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:59.074465 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:59.074465 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:59.074465 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:59.074465 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:59.074465 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:59.074465 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:59.074465 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:59.074465 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:59.074465 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:59.074465 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:59.075451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:59.075451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:59.075451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 13 12:57:59.075451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 13 12:57:59.075451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 13 12:57:59.075451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 13 12:57:59.075451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 13 12:57:59.075451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 13 12:57:59.075451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 13 12:57:59.075451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:59.075451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 278/557] 10.222.0.23 () {56 vars in 1117 bytes} [Thu Oct 13 12:57:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:59.086449 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad27c3-60db-408d-a43e-c114903a592e None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:59.136743 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:59.154270 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:59.227270 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb789aeb-ec15-41cf-ab44-d4fe836a7773 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:59.256554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad27c3-60db-408d-a43e-c114903a592e None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:59.281981 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af72bed6-4a28-4994-b44e-a6d956b9eb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:59.282630 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72bed6-4a28-4994-b44e-a6d956b9eb27 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:59.282914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72bed6-4a28-4994-b44e-a6d956b9eb27 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:59.283280 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72bed6-4a28-4994-b44e-a6d956b9eb27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:59.296831 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3472ae-20cc-4e63-bcd3-f924d4ff65a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:59.297728 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3472ae-20cc-4e63-bcd3-f924d4ff65a7 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:59.298176 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3472ae-20cc-4e63-bcd3-f924d4ff65a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:59.298625 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3472ae-20cc-4e63-bcd3-f924d4ff65a7 None admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:59.299560 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3472ae-20cc-4e63-bcd3-f924d4ff65a7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:59.300035 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3472ae-20cc-4e63-bcd3-f924d4ff65a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:59.301666 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3472ae-20cc-4e63-bcd3-f924d4ff65a7 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:59.308481 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af72bed6-4a28-4994-b44e-a6d956b9eb27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:57:59.317847 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-af72bed6-4a28-4994-b44e-a6d956b9eb27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:57:59.317847 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:59.317847 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:59.317847 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:59.317847 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:59.317847 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:59.317847 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:59.317847 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:59.317847 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:59.317847 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:59.317847 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:59.317847 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:59.317847 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:57:59.317847 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:57:59.317847 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:57:59.317847 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:57:59.318763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:57:59.318763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:57:59.318763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:57:59.318763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:57:59.318763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:59.318763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:59.318763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:59.318763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:59.318763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:59.318763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:59.318763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:59.318763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:59.318763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:59.318763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:59.318763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:59.318763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:59.319705 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:59.319705 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:59.319705 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:59.319705 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:59.319705 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:57:59.319705 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:57:59.319705 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:57:59.319705 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:57:59.319705 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:57:59.319705 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:57:59.319705 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:57:59.319705 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:59.325913 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 280/558] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 13 12:57:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:59.329391 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 279/559] 10.222.0.23 () {56 vars in 1149 bytes} [Thu Oct 13 12:57:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:59.331893 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af72bed6-4a28-4994-b44e-a6d956b9eb27 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:59.335489 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3472ae-20cc-4e63-bcd3-f924d4ff65a7 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:59.460126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:59.498684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3472ae-20cc-4e63-bcd3-f924d4ff65a7 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:59.522597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:59.529481 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af72bed6-4a28-4994-b44e-a6d956b9eb27 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:59.561863 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee6d17-ceb7-4692-9253-24a72d40c41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:59.563041 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee6d17-ceb7-4692-9253-24a72d40c41b None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:59.563541 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee6d17-ceb7-4692-9253-24a72d40c41b None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:59.564452 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee6d17-ceb7-4692-9253-24a72d40c41b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:59.585735 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ee6d17-ceb7-4692-9253-24a72d40c41b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:59.587573 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ee6d17-ceb7-4692-9253-24a72d40c41b None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:59.604148 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-d7ee6d17-ceb7-4692-9253-24a72d40c41b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:59.604148 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:59.604148 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:59.604148 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:59.604148 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:59.604148 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:59.604148 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:59.604148 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:59.604148 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:59.604148 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:59.604148 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:59.604148 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:59.604148 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:59.604148 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:59.604148 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:59.604148 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:59.605154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:59.605154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:59.605154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:59.605154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:59.605154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:59.605154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:59.605154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:59.605154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:59.605154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:59.605154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:59.605154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:59.605154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:59.605154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:59.605154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:59.605154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:59.605154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:59.605154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:59.606030 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:59.606030 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:59.606030 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:57:59.606198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cddfbbe-8d01-4324-ad87-35bd42df75b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:59.606198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cddfbbe-8d01-4324-ad87-35bd42df75b9 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:59.606198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cddfbbe-8d01-4324-ad87-35bd42df75b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:59.606198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cddfbbe-8d01-4324-ad87-35bd42df75b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:59.606474 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cddfbbe-8d01-4324-ad87-35bd42df75b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:59.606474 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cddfbbe-8d01-4324-ad87-35bd42df75b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:59.607133 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 280/560] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:59.607730 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cddfbbe-8d01-4324-ad87-35bd42df75b9 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:59.619179 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee6d17-ceb7-4692-9253-24a72d40c41b None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:59.631262 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 281/561] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:57:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:57:59.632146 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cddfbbe-8d01-4324-ad87-35bd42df75b9 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:59.742203 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cddfbbe-8d01-4324-ad87-35bd42df75b9 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:59.780539 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b6e311-0901-4a02-8121-c6d5ce3a44f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:59.781100 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6e311-0901-4a02-8121-c6d5ce3a44f3 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:59.781433 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6e311-0901-4a02-8121-c6d5ce3a44f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:59.781709 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6e311-0901-4a02-8121-c6d5ce3a44f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:59.797970 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b6e311-0901-4a02-8121-c6d5ce3a44f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:59.797970 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b6e311-0901-4a02-8121-c6d5ce3a44f3 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:59.824907 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-74b6e311-0901-4a02-8121-c6d5ce3a44f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:59.824907 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:57:59.824907 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:57:59.824907 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:57:59.824907 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:57:59.824907 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:57:59.824907 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:57:59.824907 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:57:59.824907 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:57:59.824907 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:57:59.824907 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:57:59.824907 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:57:59.824907 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:57:59.824907 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:57:59.824907 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:57:59.824907 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:57:59.825896 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:57:59.825896 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:57:59.825896 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:57:59.825896 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:57:59.825896 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:57:59.825896 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:57:59.825896 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:57:59.825896 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:57:59.825896 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:57:59.825896 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:57:59.825896 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:57:59.825896 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:57:59.825896 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:57:59.825896 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:57:59.825896 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:57:59.825896 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:57:59.825896 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:57:59.826826 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:57:59.826826 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:57:59.826826 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:57:59.830650 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 282/562] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:57:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:57:59.838651 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b6e311-0901-4a02-8121-c6d5ce3a44f3 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:57:59.853515 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee6d17-ceb7-4692-9253-24a72d40c41b None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:57:59.861926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:59.907383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:57:59.922669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa7e54-3f1f-4e1e-804b-39c12a14622b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:57:59.923306 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa7e54-3f1f-4e1e-804b-39c12a14622b None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:57:59.923636 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa7e54-3f1f-4e1e-804b-39c12a14622b None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:57:59.923912 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa7e54-3f1f-4e1e-804b-39c12a14622b None admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:57:59.924567 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa7e54-3f1f-4e1e-804b-39c12a14622b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:57:59.924871 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa7e54-3f1f-4e1e-804b-39c12a14622b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:57:59.926242 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa7e54-3f1f-4e1e-804b-39c12a14622b None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:57:59.940041 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 281/563] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:57:59.948096 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa7e54-3f1f-4e1e-804b-39c12a14622b None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:00.024183 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b6e311-0901-4a02-8121-c6d5ce3a44f3 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:00.095971 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11400893-8ae7-4c70-a076-0a6481e189dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:00.096579 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11400893-8ae7-4c70-a076-0a6481e189dd None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:00.096854 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11400893-8ae7-4c70-a076-0a6481e189dd None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:00.097123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11400893-8ae7-4c70-a076-0a6481e189dd None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:00.097768 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11400893-8ae7-4c70-a076-0a6481e189dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:00.098072 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11400893-8ae7-4c70-a076-0a6481e189dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:00.099458 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11400893-8ae7-4c70-a076-0a6481e189dd None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:00.103351 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa7e54-3f1f-4e1e-804b-39c12a14622b None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:00.125400 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 283/564] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:00.129948 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11400893-8ae7-4c70-a076-0a6481e189dd None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:00.172654 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b454f1a-020d-4b94-9c97-3dba7e27992a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:00.173308 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b454f1a-020d-4b94-9c97-3dba7e27992a None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:00.173586 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b454f1a-020d-4b94-9c97-3dba7e27992a None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:00.173856 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b454f1a-020d-4b94-9c97-3dba7e27992a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:00.192106 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b454f1a-020d-4b94-9c97-3dba7e27992a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:00.193842 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b454f1a-020d-4b94-9c97-3dba7e27992a None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:00.216753 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-7b454f1a-020d-4b94-9c97-3dba7e27992a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:58:00.216753 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:00.216753 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:00.216753 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:00.216753 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:00.216753 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:00.216753 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:00.216753 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:00.216753 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:00.216753 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:00.216753 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:00.216753 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:00.216753 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 13 12:58:00.216753 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 13 12:58:00.216753 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 13 12:58:00.217631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 13 12:58:00.217631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:00.217631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:00.217631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:00.217631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:00.217631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:00.217631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:00.217631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:00.217631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:00.217631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:00.217631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:00.217631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:00.217631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:00.217631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:00.217631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:00.217631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:00.217631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:00.218494 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 13 12:58:00.218494 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 13 12:58:00.218494 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 13 12:58:00.218494 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 13 12:58:00.218494 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 13 12:58:00.218494 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 13 12:58:00.218494 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:58:00.218494 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:00.228497 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 282/565] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:57:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:00.230538 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:00.235117 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b454f1a-020d-4b94-9c97-3dba7e27992a None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:00.247624 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11400893-8ae7-4c70-a076-0a6481e189dd None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:00.299300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19509b0-c251-4295-bfa6-86db772f37ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:00.300565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19509b0-c251-4295-bfa6-86db772f37ca None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:00.301455 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19509b0-c251-4295-bfa6-86db772f37ca None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:00.301925 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19509b0-c251-4295-bfa6-86db772f37ca None admin] PATH_INFO: `/v3/users/nova` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:00.310121 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19509b0-c251-4295-bfa6-86db772f37ca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:00.312240 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19509b0-c251-4295-bfa6-86db772f37ca None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:00.325367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-f19509b0-c251-4295-bfa6-86db772f37ca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 13 12:58:00.325367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:00.325367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:00.325367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:00.325367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:00.325367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:00.325367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:00.325367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:00.325367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:00.325367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:00.325367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:00.325367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:00.325367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 13 12:58:00.325367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 13 12:58:00.325367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 13 12:58:00.325367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 13 12:58:00.326326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:00.326326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:00.326326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 13 12:58:00.326326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:58:00.326326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 13 12:58:00.326326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:58:00.326326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:00.326326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:00.326326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:00.326326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:00.326326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:00.326326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:00.326326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:00.326326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:00.326326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:00.326326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:00.326326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:00.326326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:00.327428 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:00.327428 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:00.327428 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 13 12:58:00.327428 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 13 12:58:00.327428 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 13 12:58:00.327428 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 13 12:58:00.327428 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 13 12:58:00.327428 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 13 12:58:00.327428 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 13 12:58:00.327428 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:00.329296 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 284/566] 10.222.0.23 () {56 vars in 1093 bytes} [Thu Oct 13 12:58:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:00.340010 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19509b0-c251-4295-bfa6-86db772f37ca None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:00.345799 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:00.401228 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b454f1a-020d-4b94-9c97-3dba7e27992a None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:00.441469 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea668788-fc53-4c6f-b62d-b743d7b5f861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:00.442491 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea668788-fc53-4c6f-b62d-b743d7b5f861 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:00.442984 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea668788-fc53-4c6f-b62d-b743d7b5f861 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:00.443469 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea668788-fc53-4c6f-b62d-b743d7b5f861 None admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:00.444747 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea668788-fc53-4c6f-b62d-b743d7b5f861 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:00.445479 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea668788-fc53-4c6f-b62d-b743d7b5f861 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:00.448130 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea668788-fc53-4c6f-b62d-b743d7b5f861 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:00.462339 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 283/567] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 13 12:58:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:58:00.467776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea668788-fc53-4c6f-b62d-b743d7b5f861 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:00.518503 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19509b0-c251-4295-bfa6-86db772f37ca None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:00.601623 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520068ce-2cef-4e55-98b2-d36656691926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:00.602278 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520068ce-2cef-4e55-98b2-d36656691926 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:00.602553 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520068ce-2cef-4e55-98b2-d36656691926 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:00.602770 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520068ce-2cef-4e55-98b2-d36656691926 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:00.603536 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520068ce-2cef-4e55-98b2-d36656691926 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:00.603873 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520068ce-2cef-4e55-98b2-d36656691926 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:00.605846 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520068ce-2cef-4e55-98b2-d36656691926 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:00.611353 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:00.632260 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea668788-fc53-4c6f-b62d-b743d7b5f861 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:00.642309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 285/568] 10.222.0.23 () {56 vars in 1133 bytes} [Thu Oct 13 12:58:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:00.646831 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:00.647940 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520068ce-2cef-4e55-98b2-d36656691926 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:00.693250 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802ff42-26d8-4a41-8542-f33d67a63d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:00.693912 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802ff42-26d8-4a41-8542-f33d67a63d3e None admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:00.694187 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802ff42-26d8-4a41-8542-f33d67a63d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:00.694459 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802ff42-26d8-4a41-8542-f33d67a63d3e None admin] PATH_INFO: `/v3/projects/02308892635844b088ecdcb5413a21a0/users/b6ae674f0efc445fa970fca64e13893a/roles/b493ef4ee9d44f449e08251ae75a77d8` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:00.725410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2802ff42-26d8-4a41-8542-f33d67a63d3e None admin] RBAC: Authorizing `identity:create_grant(project_id=02308892635844b088ecdcb5413a21a0, user_id=b6ae674f0efc445fa970fca64e13893a, role_id=b493ef4ee9d44f449e08251ae75a77d8)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:00.726995 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2802ff42-26d8-4a41-8542-f33d67a63d3e None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:00.748069 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 284/569] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 13 12:58:00 2022] PUT /identity/v3/projects/02308892635844b088ecdcb5413a21a0/users/b6ae674f0efc445fa970fca64e13893a/roles/b493ef4ee9d44f449e08251ae75a77d8 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 12:58:00.806403 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520068ce-2cef-4e55-98b2-d36656691926 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:00.844874 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c8b94-919a-4bd8-8056-2d8ee43ec8cf None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:00.845211 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c8b94-919a-4bd8-8056-2d8ee43ec8cf None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:00.845424 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c8b94-919a-4bd8-8056-2d8ee43ec8cf None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:00.847024 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 285/570] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 13 12:58:00.852007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5aa95e-6cf2-4a08-8ad7-7eabe26b1511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:00.852425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046dc45-916b-4d04-abde-e1e3d054144e None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:00.852704 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046dc45-916b-4d04-abde-e1e3d054144e None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:00.852973 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046dc45-916b-4d04-abde-e1e3d054144e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:00.853464 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5aa95e-6cf2-4a08-8ad7-7eabe26b1511 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:00.853915 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5aa95e-6cf2-4a08-8ad7-7eabe26b1511 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:00.854916 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5aa95e-6cf2-4a08-8ad7-7eabe26b1511 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:00.888393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-d046dc45-916b-4d04-abde-e1e3d054144e None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:00.889836 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-d046dc45-916b-4d04-abde-e1e3d054144e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:00.927032 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:00.953324 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5aa95e-6cf2-4a08-8ad7-7eabe26b1511 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:00.954793 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5aa95e-6cf2-4a08-8ad7-7eabe26b1511 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:00.988052 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 286/571] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 13 12:58:01.086061 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 286/572] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 13 12:58:00 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 13 12:58:01.091775 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046dc45-916b-4d04-abde-e1e3d054144e None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:01.192477 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:01.207871 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046dc45-916b-4d04-abde-e1e3d054144e None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:01.248487 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d9603-3bf4-4f68-a2cf-79e2f02fca59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:01.249122 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d9603-3bf4-4f68-a2cf-79e2f02fca59 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:01.249400 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d9603-3bf4-4f68-a2cf-79e2f02fca59 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:01.249671 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d9603-3bf4-4f68-a2cf-79e2f02fca59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:01.260237 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d9603-3bf4-4f68-a2cf-79e2f02fca59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:01.263139 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d9603-3bf4-4f68-a2cf-79e2f02fca59 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:01.279669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-950d9603-3bf4-4f68-a2cf-79e2f02fca59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:58:01.279669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:01.279669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:01.279669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:01.279669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:01.279669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:01.279669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:01.279669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:01.279669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:01.279669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:01.279669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:01.279669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:01.279669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:58:01.279669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:58:01.279669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:58:01.279669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:58:01.280760 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:01.280760 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:01.280760 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:01.280760 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:01.280760 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:01.280760 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:01.280760 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:01.280760 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:01.280760 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:01.280760 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:01.280760 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:01.280760 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:01.280760 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:01.280760 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:01.280760 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:01.280760 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:01.280760 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:58:01.281679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:58:01.281679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:58:01.281679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:01.281679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 287/573] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:58:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:01.286670 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5aa95e-6cf2-4a08-8ad7-7eabe26b1511 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:01.350208 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d9603-3bf4-4f68-a2cf-79e2f02fca59 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:01.409205 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:01.425296 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5aa95e-6cf2-4a08-8ad7-7eabe26b1511 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:01.494013 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf49b5-db8e-4bc2-a164-441d2996d582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:01.494544 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf49b5-db8e-4bc2-a164-441d2996d582 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:01.495631 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf49b5-db8e-4bc2-a164-441d2996d582 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:01.495928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf49b5-db8e-4bc2-a164-441d2996d582 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:01.496576 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf49b5-db8e-4bc2-a164-441d2996d582 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:01.496882 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf49b5-db8e-4bc2-a164-441d2996d582 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:01.498243 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf49b5-db8e-4bc2-a164-441d2996d582 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:01.510403 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 287/574] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:58:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:01.515841 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf49b5-db8e-4bc2-a164-441d2996d582 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:01.557233 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d9603-3bf4-4f68-a2cf-79e2f02fca59 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:01.606562 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbace22-481d-4b21-9efd-782b7be4aaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:01.607318 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbace22-481d-4b21-9efd-782b7be4aaf4 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:01.607593 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbace22-481d-4b21-9efd-782b7be4aaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:01.607857 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbace22-481d-4b21-9efd-782b7be4aaf4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:01.608437 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbace22-481d-4b21-9efd-782b7be4aaf4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:01.609901 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbace22-481d-4b21-9efd-782b7be4aaf4 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:01.619206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-8cbace22-481d-4b21-9efd-782b7be4aaf4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 13 12:58:01.619206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:01.619206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 13 12:58:01.619206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 13 12:58:01.619206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 13 12:58:01.619206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 13 12:58:01.619206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 13 12:58:01.619206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:01.619206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 13 12:58:01.619206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:01.619206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 12:58:01.619206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:01.619206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:01.619206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:01.619206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:01.619206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:01.619206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:01.620253 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:01.620253 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:01.620253 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:01.620253 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:01.620253 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:01.620253 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:01.620253 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 13 12:58:01.620253 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 13 12:58:01.620253 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 13 12:58:01.620253 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 13 12:58:01.620253 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:01.620253 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:01.620253 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:01.620253 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:01.620253 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:01.620253 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:01.621079 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:01.621079 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:01.621079 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:01.621079 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:01.621079 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:01.621079 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:01.621079 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:01.621079 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:01.621079 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:01.621079 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:01.621079 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 13 12:58:01.621079 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:01.621079 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 13 12:58:01.621079 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:01.625250 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 288/575] 10.222.0.23 () {56 vars in 1129 bytes} [Thu Oct 13 12:58:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:58:01.625462 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf49b5-db8e-4bc2-a164-441d2996d582 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:01.626628 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73256b-65ca-484a-8264-dca799eaae55 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:01.627006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73256b-65ca-484a-8264-dca799eaae55 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:01.627308 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73256b-65ca-484a-8264-dca799eaae55 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:01.628920 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 289/576] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:01.629620 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73256b-65ca-484a-8264-dca799eaae55 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:01.637504 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:01.662267 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa40c4-af5b-49a3-acb4-5f1e8c5824ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:01.662935 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa40c4-af5b-49a3-acb4-5f1e8c5824ff None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:01.663243 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa40c4-af5b-49a3-acb4-5f1e8c5824ff None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:01.663519 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa40c4-af5b-49a3-acb4-5f1e8c5824ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:01.670890 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fa40c4-af5b-49a3-acb4-5f1e8c5824ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:01.672435 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fa40c4-af5b-49a3-acb4-5f1e8c5824ff None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:01.683829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-18fa40c4-af5b-49a3-acb4-5f1e8c5824ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:58:01.683829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:01.683829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:01.683829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:01.683829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:01.683829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:01.683829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:01.683829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:01.683829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:01.683829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:01.683829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:01.683829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:01.683829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 13 12:58:01.683829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 13 12:58:01.683829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 13 12:58:01.685861 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 13 12:58:01.685861 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:01.685861 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:01.685861 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:01.685861 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:01.685861 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:01.685861 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:01.685861 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:01.685861 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:01.685861 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:01.685861 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:01.685861 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:01.685861 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:01.685861 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:01.685861 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:01.685861 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:01.685861 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:01.688331 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 13 12:58:01.688331 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 13 12:58:01.688331 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 13 12:58:01.688331 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 13 12:58:01.688331 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 13 12:58:01.688331 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 13 12:58:01.688331 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:58:01.688331 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:01.688331 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 288/577] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:58:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:58:01.688331 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7070e-d231-4a28-a592-b5b41c4b1f77 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:01.688331 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7070e-d231-4a28-a592-b5b41c4b1f77 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:01.695085 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7070e-d231-4a28-a592-b5b41c4b1f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:01.710339 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-7fa7070e-d231-4a28-a592-b5b41c4b1f77 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:01.711539 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7fa7070e-d231-4a28-a592-b5b41c4b1f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:01.719782 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:01.749890 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73256b-65ca-484a-8264-dca799eaae55 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:01.787295 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 289/578] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:58:01.787295 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa7070e-d231-4a28-a592-b5b41c4b1f77 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:01.795789 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6cbcf0-1923-47fa-af92-5388f601857b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:01.796450 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6cbcf0-1923-47fa-af92-5388f601857b None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:01.796724 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6cbcf0-1923-47fa-af92-5388f601857b None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:01.797042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6cbcf0-1923-47fa-af92-5388f601857b None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:01.797672 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6cbcf0-1923-47fa-af92-5388f601857b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:01.797970 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6cbcf0-1923-47fa-af92-5388f601857b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:01.801342 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6cbcf0-1923-47fa-af92-5388f601857b None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:01.808669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 290/579] 10.222.0.23 () {56 vars in 1124 bytes} [Thu Oct 13 12:58:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:01.810564 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26394a7-5c49-45b7-9896-d3e09f87c568 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:01.811092 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26394a7-5c49-45b7-9896-d3e09f87c568 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:01.811995 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26394a7-5c49-45b7-9896-d3e09f87c568 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:01.814522 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 291/580] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:01.815489 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26394a7-5c49-45b7-9896-d3e09f87c568 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:01.894550 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:01.920666 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa7070e-d231-4a28-a592-b5b41c4b1f77 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:01.947365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26394a7-5c49-45b7-9896-d3e09f87c568 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:01.959539 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66e795-0c18-4793-9812-13911f3f253e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:01.960240 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66e795-0c18-4793-9812-13911f3f253e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:01.960578 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66e795-0c18-4793-9812-13911f3f253e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:01.960915 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66e795-0c18-4793-9812-13911f3f253e None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:01.961751 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66e795-0c18-4793-9812-13911f3f253e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:01.962374 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66e795-0c18-4793-9812-13911f3f253e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:01.964682 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66e795-0c18-4793-9812-13911f3f253e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:01.982829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 290/581] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 13 12:58:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:01.982829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c7c01-a0b8-4d95-872c-4a39ed47e0b8 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:01.982829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c7c01-a0b8-4d95-872c-4a39ed47e0b8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:01.982829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c7c01-a0b8-4d95-872c-4a39ed47e0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:01.987245 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:02.001066 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f73c7c01-a0b8-4d95-872c-4a39ed47e0b8 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:02.002273 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f73c7c01-a0b8-4d95-872c-4a39ed47e0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:02.003036 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5b9a6-606f-4aa4-b5ea-7b18fa2fee6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:02.003863 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5b9a6-606f-4aa4-b5ea-7b18fa2fee6d None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:02.004315 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5b9a6-606f-4aa4-b5ea-7b18fa2fee6d None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:02.004959 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5b9a6-606f-4aa4-b5ea-7b18fa2fee6d None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:02.006658 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b5b9a6-606f-4aa4-b5ea-7b18fa2fee6d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:02.007194 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b5b9a6-606f-4aa4-b5ea-7b18fa2fee6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:02.008735 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b5b9a6-606f-4aa4-b5ea-7b18fa2fee6d None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:02.017563 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 292/582] 10.222.0.23 () {56 vars in 1124 bytes} [Thu Oct 13 12:58:01 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:02.019042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5b9a6-606f-4aa4-b5ea-7b18fa2fee6d None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:02.074589 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 291/583] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:58:02.078959 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c7c01-a0b8-4d95-872c-4a39ed47e0b8 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:02.134501 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5b9a6-606f-4aa4-b5ea-7b18fa2fee6d None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:02.164446 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:02.174686 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6aba7-300f-42bb-8005-7d7b81997c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:02.175621 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6aba7-300f-42bb-8005-7d7b81997c34 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:02.176159 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6aba7-300f-42bb-8005-7d7b81997c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:02.176637 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6aba7-300f-42bb-8005-7d7b81997c34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:02.177568 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d6aba7-300f-42bb-8005-7d7b81997c34 None admin] RBAC: Adding query filter params (role.id=70af94fc6a8149f2a38f74b843e1a9f0, scope.project.id=02308892635844b088ecdcb5413a21a0, user.id=12700d985a724595af630c81e63f86d7) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:02.178080 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d6aba7-300f-42bb-8005-7d7b81997c34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:02.180254 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d6aba7-300f-42bb-8005-7d7b81997c34 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:02.189581 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 293/584] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 13 12:58:02 2022] GET /identity/v3/role_assignments?user.id=12700d985a724595af630c81e63f86d7&scope.project.id=02308892635844b088ecdcb5413a21a0&role.id=70af94fc6a8149f2a38f74b843e1a9f0 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:02.190965 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6aba7-300f-42bb-8005-7d7b81997c34 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:02.207842 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c7c01-a0b8-4d95-872c-4a39ed47e0b8 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:02.240254 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:02.248228 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858856c-f7ff-4164-ab8b-bf134958d213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:02.248945 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858856c-f7ff-4164-ab8b-bf134958d213 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:02.249318 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858856c-f7ff-4164-ab8b-bf134958d213 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:02.249657 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858856c-f7ff-4164-ab8b-bf134958d213 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:02.250750 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d858856c-f7ff-4164-ab8b-bf134958d213 None admin] RBAC: Adding query filter params (interface=public, service_id=2d3f5d0d32f74fb4b061e590e2304160, region_id=RegionOne) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:02.250750 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d858856c-f7ff-4164-ab8b-bf134958d213 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:02.252373 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d858856c-f7ff-4164-ab8b-bf134958d213 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:02.262698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 292/585] 10.222.0.23 () {56 vars in 1251 bytes} [Thu Oct 13 12:58:02 2022] GET /identity/v3/endpoints?service_id=2d3f5d0d32f74fb4b061e590e2304160&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:02.276711 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858856c-f7ff-4164-ab8b-bf134958d213 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:02.337476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6aba7-300f-42bb-8005-7d7b81997c34 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:02.381352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e867815-2ecf-46b9-8017-f944e0b8dcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:02.382807 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e867815-2ecf-46b9-8017-f944e0b8dcec None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:02.383376 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e867815-2ecf-46b9-8017-f944e0b8dcec None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:02.383811 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e867815-2ecf-46b9-8017-f944e0b8dcec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:02.399297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e867815-2ecf-46b9-8017-f944e0b8dcec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:58:02.405282 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-0e867815-2ecf-46b9-8017-f944e0b8dcec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:58:02.405282 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:02.405282 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:02.405282 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:02.405282 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:02.405282 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:02.405282 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:02.405282 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:02.405282 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:02.405282 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:02.405282 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:02.405282 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:02.405282 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:58:02.405282 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:58:02.405282 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:58:02.405282 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:58:02.406490 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:58:02.406490 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:58:02.406490 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:58:02.406490 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:58:02.406490 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:02.406490 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:02.406490 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:02.406490 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:02.406490 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:02.406490 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:02.406490 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:02.406490 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:02.406490 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:02.406490 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:02.406490 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:02.406490 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:02.407669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:02.407669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:02.407669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:02.407669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:02.407669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:58:02.407669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:58:02.407669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:58:02.407669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:58:02.407669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:58:02.407669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:58:02.407669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:58:02.407669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:02.411481 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 294/586] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 13 12:58:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:02.412430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e867815-2ecf-46b9-8017-f944e0b8dcec None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:02.479283 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858856c-f7ff-4164-ab8b-bf134958d213 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:02.492853 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:02.516283 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a046bb-f4f7-4450-b7ea-16418634521f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:02.517172 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a046bb-f4f7-4450-b7ea-16418634521f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:02.517633 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a046bb-f4f7-4450-b7ea-16418634521f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:02.518233 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a046bb-f4f7-4450-b7ea-16418634521f None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:02.519016 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a046bb-f4f7-4450-b7ea-16418634521f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:02.520746 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a046bb-f4f7-4450-b7ea-16418634521f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:02.527364 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 293/587] 10.222.0.23 () {56 vars in 1087 bytes} [Thu Oct 13 12:58:02 2022] GET /identity/v3/services => generated 772 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:02.533525 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a046bb-f4f7-4450-b7ea-16418634521f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:02.548115 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e867815-2ecf-46b9-8017-f944e0b8dcec None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:02.597622 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd47037-ed55-48b2-8166-9595f0df79f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:02.598541 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd47037-ed55-48b2-8166-9595f0df79f4 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:02.598897 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd47037-ed55-48b2-8166-9595f0df79f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:02.599273 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd47037-ed55-48b2-8166-9595f0df79f4 None admin] PATH_INFO: `/v3/services/network` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:02.599941 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd47037-ed55-48b2-8166-9595f0df79f4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:02.603627 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd47037-ed55-48b2-8166-9595f0df79f4 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:02.614662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-8dd47037-ed55-48b2-8166-9595f0df79f4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 13 12:58:02.614662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:02.614662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 13 12:58:02.614662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 13 12:58:02.614662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 13 12:58:02.614662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 13 12:58:02.614662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 13 12:58:02.614662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:02.614662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 13 12:58:02.614662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:02.614662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 12:58:02.614662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:02.614662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:02.614662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:02.614662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:02.614662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:02.614662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:02.615782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:02.615782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:02.615782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:02.615782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:02.615782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:02.615782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:02.615782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 13 12:58:02.615782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 13 12:58:02.615782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 13 12:58:02.615782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 13 12:58:02.615782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:02.615782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:02.615782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:02.615782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:02.615782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:02.615782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:02.616625 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:02.616625 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:02.616625 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:02.616625 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:02.616625 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:02.616625 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:02.616625 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:02.616625 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:02.616625 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:02.616625 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:02.616625 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 13 12:58:02.616625 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:02.616625 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 13 12:58:02.616625 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:02.621682 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 295/588] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:58:02 2022] GET /identity/v3/services/network => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:02.623099 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd47037-ed55-48b2-8166-9595f0df79f4 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:02.659516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:02.745837 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a046bb-f4f7-4450-b7ea-16418634521f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:02.805387 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd47037-ed55-48b2-8166-9595f0df79f4 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:02.810940 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260bd6d-79f0-4799-bb4a-97bd1fc78788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:02.812038 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260bd6d-79f0-4799-bb4a-97bd1fc78788 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:02.812490 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260bd6d-79f0-4799-bb4a-97bd1fc78788 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:02.812957 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260bd6d-79f0-4799-bb4a-97bd1fc78788 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:02.813784 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7260bd6d-79f0-4799-bb4a-97bd1fc78788 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:02.814720 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7260bd6d-79f0-4799-bb4a-97bd1fc78788 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:02.816788 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7260bd6d-79f0-4799-bb4a-97bd1fc78788 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:02.828803 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 294/589] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:58:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:02.836002 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260bd6d-79f0-4799-bb4a-97bd1fc78788 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:02.839015 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:02.864961 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1b69d-a56b-4376-b140-9d89b741e2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:02.865629 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1b69d-a56b-4376-b140-9d89b741e2a4 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:02.865911 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1b69d-a56b-4376-b140-9d89b741e2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:02.866181 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1b69d-a56b-4376-b140-9d89b741e2a4 None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:02.866823 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff1b69d-a56b-4376-b140-9d89b741e2a4 None admin] RBAC: Adding query filter params (name=network) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:02.868054 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff1b69d-a56b-4376-b140-9d89b741e2a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:02.869508 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff1b69d-a56b-4376-b140-9d89b741e2a4 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:02.878457 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 296/590] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Oct 13 12:58:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:02.881508 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1b69d-a56b-4376-b140-9d89b741e2a4 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:02.981358 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260bd6d-79f0-4799-bb4a-97bd1fc78788 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:03.005679 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:03.019384 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784b959-164d-478f-bf73-984b2640999e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:03.020386 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784b959-164d-478f-bf73-984b2640999e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:03.020913 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784b959-164d-478f-bf73-984b2640999e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:03.021397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784b959-164d-478f-bf73-984b2640999e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:03.032298 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4784b959-164d-478f-bf73-984b2640999e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:03.034091 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4784b959-164d-478f-bf73-984b2640999e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:03.049583 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-4784b959-164d-478f-bf73-984b2640999e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:58:03.049583 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:03.049583 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:03.049583 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:03.049583 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:03.049583 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:03.049583 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:03.049583 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:03.049583 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:03.049583 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:03.049583 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:03.049583 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:03.049583 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:58:03.049583 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:58:03.049583 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:58:03.049583 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:58:03.050585 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:03.050585 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:03.050585 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:03.050585 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:03.050585 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:03.050585 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:03.050585 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:03.050585 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:03.050585 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:03.050585 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:03.050585 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:03.050585 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:03.050585 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:03.050585 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:03.050585 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:03.050585 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:03.050585 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:58:03.051519 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:58:03.051519 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:58:03.051519 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:03.057815 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 295/591] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:58:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:03.059667 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1b69d-a56b-4376-b140-9d89b741e2a4 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:03.060192 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784b959-164d-478f-bf73-984b2640999e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:03.101074 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32680d-517f-40f7-b640-8846a2c32cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:03.101686 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32680d-517f-40f7-b640-8846a2c32cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:03.101967 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32680d-517f-40f7-b640-8846a2c32cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:03.102237 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32680d-517f-40f7-b640-8846a2c32cc2 None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:03.102875 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32680d-517f-40f7-b640-8846a2c32cc2 None admin] RBAC: Adding query filter params (type=network) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:03.103211 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32680d-517f-40f7-b640-8846a2c32cc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:03.105138 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32680d-517f-40f7-b640-8846a2c32cc2 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:03.111007 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 297/592] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Oct 13 12:58:02 2022] GET /identity/v3/services?type=network => generated 350 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:03.116744 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32680d-517f-40f7-b640-8846a2c32cc2 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:03.214243 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:03.273877 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784b959-164d-478f-bf73-984b2640999e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:03.311531 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32680d-517f-40f7-b640-8846a2c32cc2 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:03.313511 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab8ea95-85c9-41f7-936a-92288b17b559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:03.314116 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab8ea95-85c9-41f7-936a-92288b17b559 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:03.314393 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab8ea95-85c9-41f7-936a-92288b17b559 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:03.314663 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab8ea95-85c9-41f7-936a-92288b17b559 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:03.315334 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab8ea95-85c9-41f7-936a-92288b17b559 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:03.315636 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab8ea95-85c9-41f7-936a-92288b17b559 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:03.316992 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab8ea95-85c9-41f7-936a-92288b17b559 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:03.333367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 296/593] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:58:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:03.333367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f948115-3ead-4fc8-b9db-7a93947aae15 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:03.333367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f948115-3ead-4fc8-b9db-7a93947aae15 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:03.333367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f948115-3ead-4fc8-b9db-7a93947aae15 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:03.333367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 297/594] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:03.340333 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f948115-3ead-4fc8-b9db-7a93947aae15 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:03.360787 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c326b5da-164b-49b9-9d8d-2ff11be667b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:03.361665 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326b5da-164b-49b9-9d8d-2ff11be667b9 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:03.361786 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326b5da-164b-49b9-9d8d-2ff11be667b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:03.362020 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326b5da-164b-49b9-9d8d-2ff11be667b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:03.362773 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c326b5da-164b-49b9-9d8d-2ff11be667b9 None admin] RBAC: Adding query filter params (interface=public, service_id=ffefca295165466fa869414053f7e5f3, region_id=RegionOne) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:03.363155 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c326b5da-164b-49b9-9d8d-2ff11be667b9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:03.364575 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c326b5da-164b-49b9-9d8d-2ff11be667b9 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:03.377566 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 298/595] 10.222.0.23 () {56 vars in 1251 bytes} [Thu Oct 13 12:58:03 2022] GET /identity/v3/endpoints?service_id=ffefca295165466fa869414053f7e5f3&interface=public®ion_id=RegionOne => generated 194 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:58:03.379015 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607399f4-2f69-4253-9121-f7cd32bd4f22 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:03.379615 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:03.379729 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607399f4-2f69-4253-9121-f7cd32bd4f22 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:03.379924 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607399f4-2f69-4253-9121-f7cd32bd4f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:03.401630 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-607399f4-2f69-4253-9121-f7cd32bd4f22 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:03.402863 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-607399f4-2f69-4253-9121-f7cd32bd4f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:03.453954 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f948115-3ead-4fc8-b9db-7a93947aae15 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:03.486627 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 299/596] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:58:03.492259 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607399f4-2f69-4253-9121-f7cd32bd4f22 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:03.498145 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182822c-382c-442f-a27f-7415aa194561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:03.498829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182822c-382c-442f-a27f-7415aa194561 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:03.499126 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182822c-382c-442f-a27f-7415aa194561 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:03.499399 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182822c-382c-442f-a27f-7415aa194561 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:03.515830 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2182822c-382c-442f-a27f-7415aa194561 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:03.517507 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2182822c-382c-442f-a27f-7415aa194561 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:03.540007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-2182822c-382c-442f-a27f-7415aa194561 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 13 12:58:03.540007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:03.540007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:03.540007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:03.540007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:03.540007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:03.540007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:03.540007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:03.540007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:03.540007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:03.540007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:03.540007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:03.540007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 13 12:58:03.540007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 13 12:58:03.540007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 13 12:58:03.540007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 13 12:58:03.541014 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:03.541014 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:03.541014 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 13 12:58:03.541014 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:58:03.541014 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 13 12:58:03.541014 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:58:03.541014 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:03.541014 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:03.541014 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:03.541014 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:03.541014 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:03.541014 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:03.541014 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:03.541014 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:03.541014 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:03.541014 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:03.541014 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:03.541014 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:03.542128 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:03.542128 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:03.542128 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 13 12:58:03.542128 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 13 12:58:03.542128 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 13 12:58:03.542128 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 13 12:58:03.542128 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 13 12:58:03.542128 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 13 12:58:03.542128 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 13 12:58:03.542128 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:03.545736 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 298/597] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 13 12:58:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:03.556182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182822c-382c-442f-a27f-7415aa194561 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:03.565157 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:03.680984 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:03.689781 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182822c-382c-442f-a27f-7415aa194561 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:03.690701 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607399f4-2f69-4253-9121-f7cd32bd4f22 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:03.724451 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53b5b8-43b8-47e3-9566-efa9d1f8ca23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:03.725052 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53b5b8-43b8-47e3-9566-efa9d1f8ca23 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:03.725327 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53b5b8-43b8-47e3-9566-efa9d1f8ca23 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:03.725593 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53b5b8-43b8-47e3-9566-efa9d1f8ca23 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:03.726304 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a53b5b8-43b8-47e3-9566-efa9d1f8ca23 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:03.726602 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a53b5b8-43b8-47e3-9566-efa9d1f8ca23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:03.728042 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a53b5b8-43b8-47e3-9566-efa9d1f8ca23 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:03.746261 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 299/598] 10.222.0.23 () {56 vars in 1135 bytes} [Thu Oct 13 12:58:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:03.753354 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53b5b8-43b8-47e3-9566-efa9d1f8ca23 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:03.755654 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8efc7a-cbc0-4f41-9786-f702a3a9087f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:03.756559 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8efc7a-cbc0-4f41-9786-f702a3a9087f None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:03.757051 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8efc7a-cbc0-4f41-9786-f702a3a9087f None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:03.757518 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8efc7a-cbc0-4f41-9786-f702a3a9087f None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:03.758434 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8efc7a-cbc0-4f41-9786-f702a3a9087f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:03.760132 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8efc7a-cbc0-4f41-9786-f702a3a9087f None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:03.766556 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 300/599] 10.222.0.23 () {56 vars in 1087 bytes} [Thu Oct 13 12:58:03 2022] GET /identity/v3/services => generated 772 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:03.788821 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8efc7a-cbc0-4f41-9786-f702a3a9087f None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:03.871085 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53b5b8-43b8-47e3-9566-efa9d1f8ca23 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:03.945571 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1252bc6-1f73-4dba-95ca-7728fd6c1348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:03.946181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1252bc6-1f73-4dba-95ca-7728fd6c1348 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:03.946456 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1252bc6-1f73-4dba-95ca-7728fd6c1348 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:03.946719 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1252bc6-1f73-4dba-95ca-7728fd6c1348 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:03.988513 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:03.991722 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8efc7a-cbc0-4f41-9786-f702a3a9087f None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:04.002084 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:04.031949 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71891d5-0b5a-4454-9468-d05e5c5a3d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:04.032955 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71891d5-0b5a-4454-9468-d05e5c5a3d1f None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:04.033435 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71891d5-0b5a-4454-9468-d05e5c5a3d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:04.033903 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71891d5-0b5a-4454-9468-d05e5c5a3d1f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:04.037563 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1252bc6-1f73-4dba-95ca-7728fd6c1348 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:04.039004 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1252bc6-1f73-4dba-95ca-7728fd6c1348 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:04.045893 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71891d5-0b5a-4454-9468-d05e5c5a3d1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:58:04.050866 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-c71891d5-0b5a-4454-9468-d05e5c5a3d1f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 13 12:58:04.050866 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:04.050866 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:04.050866 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:04.050866 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:04.050866 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:04.050866 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:04.050866 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:04.050866 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:04.050866 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:04.050866 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:04.050866 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:04.050866 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:58:04.050866 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:58:04.050866 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:58:04.050866 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:58:04.051838 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:58:04.051838 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:58:04.051838 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:58:04.051838 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:58:04.051838 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:04.051838 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:04.051838 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:04.051838 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:04.051838 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:04.051838 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:04.051838 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:04.051838 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:04.051838 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:04.051838 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:04.051838 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:04.051838 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:04.052682 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:04.052682 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:04.052682 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:04.052682 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:04.052682 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:58:04.052682 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:58:04.052682 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:58:04.052682 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:58:04.052682 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:58:04.052682 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:58:04.052682 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 13 12:58:04.052682 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:04.054931 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 301/600] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Oct 13 12:58:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:58:04.061616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71891d5-0b5a-4454-9468-d05e5c5a3d1f None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:04.142519 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 300/601] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 13 12:58:03 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 13 12:58:04.148544 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1252bc6-1f73-4dba-95ca-7728fd6c1348 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:04.171389 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71891d5-0b5a-4454-9468-d05e5c5a3d1f None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:04.206555 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:04.208773 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052578f3-aa9f-479a-a852-b50672d8b97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:04.209698 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052578f3-aa9f-479a-a852-b50672d8b97c None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:04.210309 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052578f3-aa9f-479a-a852-b50672d8b97c None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:04.210737 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052578f3-aa9f-479a-a852-b50672d8b97c None admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:04.212234 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052578f3-aa9f-479a-a852-b50672d8b97c None admin] RBAC: Adding query filter params (name=service) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:04.212680 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052578f3-aa9f-479a-a852-b50672d8b97c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:04.214240 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052578f3-aa9f-479a-a852-b50672d8b97c None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:04.223866 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 302/602] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Oct 13 12:58:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:04.236173 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052578f3-aa9f-479a-a852-b50672d8b97c None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:04.259261 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1252bc6-1f73-4dba-95ca-7728fd6c1348 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:04.301494 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:04.307875 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8942fb56-b870-4edb-b8e1-8646fd60731f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:04.309098 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8942fb56-b870-4edb-b8e1-8646fd60731f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:04.309375 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8942fb56-b870-4edb-b8e1-8646fd60731f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:04.309646 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8942fb56-b870-4edb-b8e1-8646fd60731f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:04.330038 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8942fb56-b870-4edb-b8e1-8646fd60731f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:04.331526 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8942fb56-b870-4edb-b8e1-8646fd60731f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:04.361300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-8942fb56-b870-4edb-b8e1-8646fd60731f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:58:04.361300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:04.361300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:04.361300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:04.361300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:04.361300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:04.361300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:04.361300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:04.361300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:04.361300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:04.361300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:04.361300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:04.361300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:58:04.361300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:58:04.361300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:58:04.361300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:58:04.362305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:04.362305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:04.362305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:04.362305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:04.362305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:04.362305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:04.362305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:04.362305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:04.362305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:04.362305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:04.362305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:04.362305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:04.362305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:04.362305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:04.362305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:04.362305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:04.362305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:58:04.363235 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:58:04.363235 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:58:04.363235 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:04.363831 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 301/603] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:58:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:04.369616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8942fb56-b870-4edb-b8e1-8646fd60731f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:04.439144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052578f3-aa9f-479a-a852-b50672d8b97c None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:04.512375 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de775ce3-0e72-4ad7-9cb0-c7bf2223b47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:04.513032 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de775ce3-0e72-4ad7-9cb0-c7bf2223b47c None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:04.513309 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de775ce3-0e72-4ad7-9cb0-c7bf2223b47c None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:04.513579 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de775ce3-0e72-4ad7-9cb0-c7bf2223b47c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:04.533461 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de775ce3-0e72-4ad7-9cb0-c7bf2223b47c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:04.534918 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de775ce3-0e72-4ad7-9cb0-c7bf2223b47c None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:04.564849 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8942fb56-b870-4edb-b8e1-8646fd60731f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:04.568818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-de775ce3-0e72-4ad7-9cb0-c7bf2223b47c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:58:04.568818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:04.568818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:04.568818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:04.568818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:04.568818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:04.568818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:04.568818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:04.568818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:04.568818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:04.568818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:04.568818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:04.568818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:58:04.568818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:58:04.568818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:58:04.568818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:58:04.569775 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:04.569775 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:04.569775 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:04.569775 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:04.569775 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:04.569775 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:04.569775 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:04.569775 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:04.569775 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:04.569775 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:04.569775 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:04.569775 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:04.569775 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:04.569775 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:04.569775 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:04.569775 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:04.569775 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:58:04.570675 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:58:04.570675 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:58:04.570675 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:04.570675 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 303/604] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:58:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:04.571186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:04.576363 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de775ce3-0e72-4ad7-9cb0-c7bf2223b47c None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:04.625566 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ad87e-9863-4faf-956e-75779294f9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:04.626642 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ad87e-9863-4faf-956e-75779294f9bc None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:04.627044 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ad87e-9863-4faf-956e-75779294f9bc None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:04.627352 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ad87e-9863-4faf-956e-75779294f9bc None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:04.628156 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523ad87e-9863-4faf-956e-75779294f9bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:04.628470 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523ad87e-9863-4faf-956e-75779294f9bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:04.630472 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523ad87e-9863-4faf-956e-75779294f9bc None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:04.646101 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 302/605] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:58:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:58:04.648018 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab737cf-4982-4f52-acbc-8b0267f3bd70 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:04.648452 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab737cf-4982-4f52-acbc-8b0267f3bd70 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:04.648781 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab737cf-4982-4f52-acbc-8b0267f3bd70 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:04.651230 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 303/606] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:04.652062 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab737cf-4982-4f52-acbc-8b0267f3bd70 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:04.695321 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:04.720205 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de775ce3-0e72-4ad7-9cb0-c7bf2223b47c None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:04.767591 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee860bda-2a9d-49b9-831d-1633276412af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:04.768901 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee860bda-2a9d-49b9-831d-1633276412af None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:04.769576 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee860bda-2a9d-49b9-831d-1633276412af None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:04.769988 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee860bda-2a9d-49b9-831d-1633276412af None admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:04.771020 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee860bda-2a9d-49b9-831d-1633276412af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:04.771871 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee860bda-2a9d-49b9-831d-1633276412af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:04.773910 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee860bda-2a9d-49b9-831d-1633276412af None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:04.779030 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab737cf-4982-4f52-acbc-8b0267f3bd70 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:04.789980 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 304/607] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:58:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:04.792548 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da7c5a-5020-435a-bf10-14f0961e4334 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:04.793380 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da7c5a-5020-435a-bf10-14f0961e4334 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:04.793814 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da7c5a-5020-435a-bf10-14f0961e4334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:04.820581 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ece14-ac5c-4002-a30c-3fd3b4529d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:04.820968 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-96da7c5a-5020-435a-bf10-14f0961e4334 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:04.821209 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ece14-ac5c-4002-a30c-3fd3b4529d7a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:04.821487 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ece14-ac5c-4002-a30c-3fd3b4529d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:04.821754 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ece14-ac5c-4002-a30c-3fd3b4529d7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:04.823026 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-96da7c5a-5020-435a-bf10-14f0961e4334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:04.829831 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ece14-ac5c-4002-a30c-3fd3b4529d7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:04.831445 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ece14-ac5c-4002-a30c-3fd3b4529d7a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:04.845057 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-3a6ece14-ac5c-4002-a30c-3fd3b4529d7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:58:04.845057 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:04.845057 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:04.845057 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:04.845057 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:04.845057 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:04.845057 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:04.845057 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:04.845057 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:04.845057 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:04.845057 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:04.845057 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:04.845057 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 13 12:58:04.845057 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 13 12:58:04.845057 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 13 12:58:04.846208 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 13 12:58:04.846208 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:04.846208 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:04.846208 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:04.846208 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:04.846208 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:04.846208 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:04.846208 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:04.846208 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:04.846208 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:04.846208 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:04.846208 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:04.846208 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:04.846208 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:04.846208 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:04.846208 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:04.846208 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:04.847477 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 13 12:58:04.847477 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 13 12:58:04.847477 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 13 12:58:04.847477 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 13 12:58:04.847477 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 13 12:58:04.847477 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 13 12:58:04.847477 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:58:04.847477 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:04.847477 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 304/608] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:58:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:04.847477 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ece14-ac5c-4002-a30c-3fd3b4529d7a None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:04.878934 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:04.902455 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 305/609] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:58:04.903661 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da7c5a-5020-435a-bf10-14f0961e4334 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:05.018298 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ece14-ac5c-4002-a30c-3fd3b4529d7a None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:05.046066 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:05.090809 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2df42-0c67-40a5-b28b-44b2eb6a5628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:05.091654 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2df42-0c67-40a5-b28b-44b2eb6a5628 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:05.091935 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2df42-0c67-40a5-b28b-44b2eb6a5628 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:05.092462 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2df42-0c67-40a5-b28b-44b2eb6a5628 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:05.105745 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da7c5a-5020-435a-bf10-14f0961e4334 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:05.109441 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2df42-0c67-40a5-b28b-44b2eb6a5628 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:05.115316 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2df42-0c67-40a5-b28b-44b2eb6a5628 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:05.132435 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-b2a2df42-0c67-40a5-b28b-44b2eb6a5628 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 13 12:58:05.132435 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:05.132435 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:05.132435 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:05.132435 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:05.132435 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:05.132435 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:05.132435 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:05.132435 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:05.132435 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:05.132435 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:05.132435 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:05.132435 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 13 12:58:05.132435 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 13 12:58:05.132435 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 13 12:58:05.132435 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 13 12:58:05.133426 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:05.133426 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:05.133426 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 13 12:58:05.133426 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:58:05.133426 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 13 12:58:05.133426 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:58:05.133426 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:05.133426 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:05.133426 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:05.133426 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:05.133426 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:05.133426 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:05.133426 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:05.133426 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:05.133426 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:05.133426 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:05.133426 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:05.133426 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:05.134936 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:05.134936 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:05.134936 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 13 12:58:05.134936 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 13 12:58:05.134936 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 13 12:58:05.134936 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 13 12:58:05.134936 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 13 12:58:05.134936 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 13 12:58:05.134936 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 13 12:58:05.134936 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:05.135647 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 305/610] 10.222.0.23 () {56 vars in 1117 bytes} [Thu Oct 13 12:58:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:05.140095 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2df42-0c67-40a5-b28b-44b2eb6a5628 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:05.174835 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572771a6-9cf6-4135-80cf-799e93c13948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:05.175514 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572771a6-9cf6-4135-80cf-799e93c13948 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:05.175790 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572771a6-9cf6-4135-80cf-799e93c13948 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:05.176063 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572771a6-9cf6-4135-80cf-799e93c13948 None admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:05.176758 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572771a6-9cf6-4135-80cf-799e93c13948 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:05.177063 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572771a6-9cf6-4135-80cf-799e93c13948 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:05.178499 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572771a6-9cf6-4135-80cf-799e93c13948 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:05.203566 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 306/611] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 13 12:58:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:05.212726 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572771a6-9cf6-4135-80cf-799e93c13948 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:05.263599 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:05.279960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2df42-0c67-40a5-b28b-44b2eb6a5628 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:05.313987 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b5b83-45a2-4cc7-882e-b3cb0e7fd9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:05.314592 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b5b83-45a2-4cc7-882e-b3cb0e7fd9eb None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:05.314874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b5b83-45a2-4cc7-882e-b3cb0e7fd9eb None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:05.315198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b5b83-45a2-4cc7-882e-b3cb0e7fd9eb None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:05.315920 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b5b83-45a2-4cc7-882e-b3cb0e7fd9eb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:05.316228 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b5b83-45a2-4cc7-882e-b3cb0e7fd9eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:05.317651 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b5b83-45a2-4cc7-882e-b3cb0e7fd9eb None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:05.322552 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572771a6-9cf6-4135-80cf-799e93c13948 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:05.334352 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 306/612] 10.222.0.23 () {56 vars in 1149 bytes} [Thu Oct 13 12:58:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:05.335180 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b5b83-45a2-4cc7-882e-b3cb0e7fd9eb None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:05.356023 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:05.362805 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5d51f-f425-42f7-8601-7d7c4d29612c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:05.363545 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5d51f-f425-42f7-8601-7d7c4d29612c None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:05.363935 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5d51f-f425-42f7-8601-7d7c4d29612c None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:05.364210 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5d51f-f425-42f7-8601-7d7c4d29612c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:05.365017 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5d51f-f425-42f7-8601-7d7c4d29612c None admin] RBAC: Adding query filter params (role.id=70af94fc6a8149f2a38f74b843e1a9f0, scope.project.id=02308892635844b088ecdcb5413a21a0, user.id=526000341fd146db8c661f1fd14d4801) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:05.365231 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5d51f-f425-42f7-8601-7d7c4d29612c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:05.366807 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5d51f-f425-42f7-8601-7d7c4d29612c None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:05.373183 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 307/613] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 13 12:58:05 2022] GET /identity/v3/role_assignments?user.id=526000341fd146db8c661f1fd14d4801&scope.project.id=02308892635844b088ecdcb5413a21a0&role.id=70af94fc6a8149f2a38f74b843e1a9f0 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:58:05.374469 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e0af7-00ab-4268-bbd7-0a2dc0dec370 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:05.374753 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e0af7-00ab-4268-bbd7-0a2dc0dec370 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:05.375078 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e0af7-00ab-4268-bbd7-0a2dc0dec370 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:05.379139 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 308/614] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:05.379139 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e0af7-00ab-4268-bbd7-0a2dc0dec370 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:05.462425 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b5b83-45a2-4cc7-882e-b3cb0e7fd9eb None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:05.502469 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa6937-7c77-4e4f-811f-01875b351ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:05.503481 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa6937-7c77-4e4f-811f-01875b351ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:05.503938 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa6937-7c77-4e4f-811f-01875b351ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:05.504374 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa6937-7c77-4e4f-811f-01875b351ca9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:05.505161 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa6937-7c77-4e4f-811f-01875b351ca9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:05.506707 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa6937-7c77-4e4f-811f-01875b351ca9 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:05.514326 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e0af7-00ab-4268-bbd7-0a2dc0dec370 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:05.517829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-7baa6937-7c77-4e4f-811f-01875b351ca9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 13 12:58:05.517829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:05.517829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 13 12:58:05.517829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 13 12:58:05.517829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 13 12:58:05.517829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 13 12:58:05.517829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 13 12:58:05.517829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:05.517829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 13 12:58:05.517829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:05.517829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 12:58:05.517829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:05.517829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:05.517829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:05.517829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:05.517829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:05.518775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:05.518775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:05.518775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:05.518775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:05.518775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:05.518775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:05.518775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:05.518775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 13 12:58:05.518775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 13 12:58:05.518775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 13 12:58:05.518775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 13 12:58:05.518775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:05.518775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:05.518775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:05.518775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:05.518775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:05.518775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:05.519698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:05.519698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:05.519698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:05.519698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:05.519698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:05.519698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:05.519698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:05.519698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:05.519698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:05.519698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:05.519698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 13 12:58:05.519698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:05.519698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 13 12:58:05.519698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:05.522561 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 307/615] 10.222.0.23 () {56 vars in 1132 bytes} [Thu Oct 13 12:58:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:05.523980 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ec32e-45c6-43da-a350-0189d9627611 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:05.524364 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ec32e-45c6-43da-a350-0189d9627611 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:05.524734 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ec32e-45c6-43da-a350-0189d9627611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:05.543674 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:05.552622 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f88ec32e-45c6-43da-a350-0189d9627611 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:05.554043 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f88ec32e-45c6-43da-a350-0189d9627611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:05.563236 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12356389-d2f2-4599-8dc4-8bf34a740cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:05.564008 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12356389-d2f2-4599-8dc4-8bf34a740cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:05.564387 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12356389-d2f2-4599-8dc4-8bf34a740cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:05.564754 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12356389-d2f2-4599-8dc4-8bf34a740cf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:05.638659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:05.644025 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 308/616] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:58:05.645046 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ec32e-45c6-43da-a350-0189d9627611 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:05.648082 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12356389-d2f2-4599-8dc4-8bf34a740cf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:05.649560 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12356389-d2f2-4599-8dc4-8bf34a740cf7 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:05.787027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ec32e-45c6-43da-a350-0189d9627611 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:05.795242 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 309/617] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 13 12:58:05 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 13 12:58:05.797465 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12356389-d2f2-4599-8dc4-8bf34a740cf7 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:05.860070 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12fb68-40c2-4b3f-b573-5835c7d3f927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:05.860693 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12fb68-40c2-4b3f-b573-5835c7d3f927 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:05.861015 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12fb68-40c2-4b3f-b573-5835c7d3f927 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:05.861289 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12fb68-40c2-4b3f-b573-5835c7d3f927 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:05.861925 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12fb68-40c2-4b3f-b573-5835c7d3f927 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:05.862223 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12fb68-40c2-4b3f-b573-5835c7d3f927 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:05.863633 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12fb68-40c2-4b3f-b573-5835c7d3f927 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:05.865576 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:05.874771 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 309/618] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Oct 13 12:58:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:05.875765 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12fb68-40c2-4b3f-b573-5835c7d3f927 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:05.976386 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:05.986005 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12356389-d2f2-4599-8dc4-8bf34a740cf7 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:06.021590 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726405bc-f2dc-4e00-b2eb-fe51462ac8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:06.022225 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726405bc-f2dc-4e00-b2eb-fe51462ac8a3 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:06.022504 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726405bc-f2dc-4e00-b2eb-fe51462ac8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:06.022776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726405bc-f2dc-4e00-b2eb-fe51462ac8a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:06.029299 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12fb68-40c2-4b3f-b573-5835c7d3f927 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:06.035347 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726405bc-f2dc-4e00-b2eb-fe51462ac8a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:06.036818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726405bc-f2dc-4e00-b2eb-fe51462ac8a3 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:06.055448 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-726405bc-f2dc-4e00-b2eb-fe51462ac8a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:58:06.055448 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:06.055448 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:06.055448 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:06.055448 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:06.055448 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:06.055448 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:06.055448 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:06.055448 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:06.055448 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:06.055448 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:06.055448 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:06.055448 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:58:06.055448 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:58:06.055448 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:58:06.055448 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:58:06.056476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:06.056476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:06.056476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:06.056476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:06.056476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:06.056476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:06.056476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:06.056476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:06.056476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:06.056476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:06.056476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:06.056476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:06.056476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:06.056476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:06.056476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:06.056476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:06.056476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:58:06.057439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:58:06.057439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:58:06.057439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:06.057439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 310/619] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:58:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:58:06.057688 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726405bc-f2dc-4e00-b2eb-fe51462ac8a3 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:06.071800 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf8423-8da1-44b0-bd22-f8630d00a775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:06.072524 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf8423-8da1-44b0-bd22-f8630d00a775 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:06.072843 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf8423-8da1-44b0-bd22-f8630d00a775 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:06.073115 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf8423-8da1-44b0-bd22-f8630d00a775 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:06.073745 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf8423-8da1-44b0-bd22-f8630d00a775 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:06.074048 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf8423-8da1-44b0-bd22-f8630d00a775 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:06.075561 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf8423-8da1-44b0-bd22-f8630d00a775 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:06.080822 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 310/620] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Oct 13 12:58:05 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:06.081665 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf8423-8da1-44b0-bd22-f8630d00a775 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:06.149471 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:06.193094 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf8423-8da1-44b0-bd22-f8630d00a775 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:06.233587 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96694bca-e445-4627-8e59-4aff96dad447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:06.234480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96694bca-e445-4627-8e59-4aff96dad447 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:06.234901 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96694bca-e445-4627-8e59-4aff96dad447 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:06.235275 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96694bca-e445-4627-8e59-4aff96dad447 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:06.236015 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96694bca-e445-4627-8e59-4aff96dad447 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:06.236388 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96694bca-e445-4627-8e59-4aff96dad447 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:06.237970 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96694bca-e445-4627-8e59-4aff96dad447 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:06.252921 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726405bc-f2dc-4e00-b2eb-fe51462ac8a3 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:06.253252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 311/621] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:58:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:06.274294 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96694bca-e445-4627-8e59-4aff96dad447 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:06.296143 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362a879-6aca-4873-850a-8e45f9d592ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:06.296782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362a879-6aca-4873-850a-8e45f9d592ae None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:06.297062 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362a879-6aca-4873-850a-8e45f9d592ae None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:06.297336 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362a879-6aca-4873-850a-8e45f9d592ae None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:06.297939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362a879-6aca-4873-850a-8e45f9d592ae None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:06.299384 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362a879-6aca-4873-850a-8e45f9d592ae None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:06.323554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-2362a879-6aca-4873-850a-8e45f9d592ae None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 13 12:58:06.323554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:06.323554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 13 12:58:06.323554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 13 12:58:06.323554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 13 12:58:06.323554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 13 12:58:06.323554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 13 12:58:06.323554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:06.323554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 13 12:58:06.323554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:06.323554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 12:58:06.323554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:06.323554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:06.323554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:06.323554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:06.323554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:06.323554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:06.325882 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:06.325882 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:06.325882 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:06.325882 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:06.325882 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:06.325882 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:06.325882 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 13 12:58:06.325882 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 13 12:58:06.325882 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 13 12:58:06.325882 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 13 12:58:06.325882 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:06.325882 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:06.325882 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:06.325882 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:06.325882 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:06.325882 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:06.327963 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:06.327963 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:06.327963 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:06.327963 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:06.327963 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:06.327963 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:06.327963 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:06.327963 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:06.327963 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:06.327963 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:06.327963 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 13 12:58:06.327963 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:06.327963 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 13 12:58:06.327963 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:06.327963 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 311/622] 10.222.0.23 () {56 vars in 1129 bytes} [Thu Oct 13 12:58:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:06.340353 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362a879-6aca-4873-850a-8e45f9d592ae None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:06.355041 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:06.434036 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96694bca-e445-4627-8e59-4aff96dad447 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:06.454969 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362a879-6aca-4873-850a-8e45f9d592ae None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:06.461713 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:06.480540 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e3cec6-efc8-4f6a-87eb-85757c24fa8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:06.481186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e3cec6-efc8-4f6a-87eb-85757c24fa8c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:06.481467 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e3cec6-efc8-4f6a-87eb-85757c24fa8c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:06.481739 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e3cec6-efc8-4f6a-87eb-85757c24fa8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:06.490410 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e3cec6-efc8-4f6a-87eb-85757c24fa8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:06.491918 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e3cec6-efc8-4f6a-87eb-85757c24fa8c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:06.496379 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f8910f-0249-42ff-9560-2ada7189ca5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:06.496988 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8910f-0249-42ff-9560-2ada7189ca5d None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:06.497267 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8910f-0249-42ff-9560-2ada7189ca5d None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:06.497536 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8910f-0249-42ff-9560-2ada7189ca5d None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:06.498167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f8910f-0249-42ff-9560-2ada7189ca5d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:06.498469 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f8910f-0249-42ff-9560-2ada7189ca5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:06.500035 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f8910f-0249-42ff-9560-2ada7189ca5d None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:06.504692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-37e3cec6-efc8-4f6a-87eb-85757c24fa8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:58:06.504692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:06.504692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:06.504692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:06.504692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:06.504692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:06.504692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:06.504692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:06.504692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:06.504692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:06.504692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:06.504692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:06.504692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 13 12:58:06.504692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 13 12:58:06.504692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 13 12:58:06.506698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 13 12:58:06.506698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:06.506698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:06.506698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:06.506698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:06.506698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:06.506698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:06.506698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:06.506698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:06.506698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:06.506698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:06.506698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:06.506698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:06.506698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:06.506698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:06.506698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:06.506698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:06.508516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 13 12:58:06.508516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 13 12:58:06.508516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 13 12:58:06.508516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 13 12:58:06.508516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 13 12:58:06.508516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 13 12:58:06.508516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:58:06.508516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:06.509042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 312/623] 10.222.0.23 () {56 vars in 1124 bytes} [Thu Oct 13 12:58:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:58:06.509042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b477be-4554-4ed2-90f5-f58cfe52acab None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:06.509212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 312/624] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:58:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:06.509294 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b477be-4554-4ed2-90f5-f58cfe52acab None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:06.509294 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b477be-4554-4ed2-90f5-f58cfe52acab None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:06.509294 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 313/625] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:06.513435 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e3cec6-efc8-4f6a-87eb-85757c24fa8c None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:06.513704 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60410865-5673-4597-828c-d8c509a8d4c5 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:06.513991 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60410865-5673-4597-828c-d8c509a8d4c5 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:06.514370 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60410865-5673-4597-828c-d8c509a8d4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:06.533648 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-60410865-5673-4597-828c-d8c509a8d4c5 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:06.534818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-60410865-5673-4597-828c-d8c509a8d4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:06.607395 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 314/626] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:58:06.608178 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60410865-5673-4597-828c-d8c509a8d4c5 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:06.611299 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:06.631418 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e3cec6-efc8-4f6a-87eb-85757c24fa8c None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:06.669939 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760235d0-42ee-4c71-b063-3872243b182d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:06.670858 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760235d0-42ee-4c71-b063-3872243b182d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:06.671408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760235d0-42ee-4c71-b063-3872243b182d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:06.671687 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760235d0-42ee-4c71-b063-3872243b182d None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:06.672385 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760235d0-42ee-4c71-b063-3872243b182d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:06.672690 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760235d0-42ee-4c71-b063-3872243b182d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:06.674128 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760235d0-42ee-4c71-b063-3872243b182d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:06.697558 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 313/627] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 13 12:58:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:06.703552 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760235d0-42ee-4c71-b063-3872243b182d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:06.747155 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60410865-5673-4597-828c-d8c509a8d4c5 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:06.783734 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:06.795679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66113b4e-d5aa-4f81-9438-5224eb9cd6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:06.796291 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66113b4e-d5aa-4f81-9438-5224eb9cd6f1 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:06.796566 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66113b4e-d5aa-4f81-9438-5224eb9cd6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:06.796946 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66113b4e-d5aa-4f81-9438-5224eb9cd6f1 None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:06.797543 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66113b4e-d5aa-4f81-9438-5224eb9cd6f1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:06.797843 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66113b4e-d5aa-4f81-9438-5224eb9cd6f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:06.799223 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66113b4e-d5aa-4f81-9438-5224eb9cd6f1 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:06.804240 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 315/628] 10.222.0.23 () {56 vars in 1124 bytes} [Thu Oct 13 12:58:06 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:06.810280 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66113b4e-d5aa-4f81-9438-5224eb9cd6f1 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:06.895520 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:06.896978 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760235d0-42ee-4c71-b063-3872243b182d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:06.919752 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66113b4e-d5aa-4f81-9438-5224eb9cd6f1 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:06.959596 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44064d42-a9d1-4597-9a10-7b8405abe99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:06.960456 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44064d42-a9d1-4597-9a10-7b8405abe99e None admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:06.961116 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44064d42-a9d1-4597-9a10-7b8405abe99e None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:06.961407 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44064d42-a9d1-4597-9a10-7b8405abe99e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:06.962351 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44064d42-a9d1-4597-9a10-7b8405abe99e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:06.964638 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44064d42-a9d1-4597-9a10-7b8405abe99e None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:06.968851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9732ff8-faf0-455d-88cc-ee2ef75dcb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:06.969461 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9732ff8-faf0-455d-88cc-ee2ef75dcb5e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:06.969737 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9732ff8-faf0-455d-88cc-ee2ef75dcb5e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:06.970007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9732ff8-faf0-455d-88cc-ee2ef75dcb5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:06.970729 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9732ff8-faf0-455d-88cc-ee2ef75dcb5e None admin] RBAC: Adding query filter params (role.id=b493ef4ee9d44f449e08251ae75a77d8, scope.project.id=02308892635844b088ecdcb5413a21a0, user.id=b6ae674f0efc445fa970fca64e13893a) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:06.971034 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9732ff8-faf0-455d-88cc-ee2ef75dcb5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:06.972492 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9732ff8-faf0-455d-88cc-ee2ef75dcb5e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:06.980255 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 314/629] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 13 12:58:06 2022] GET /identity/v3/role_assignments?user.id=b6ae674f0efc445fa970fca64e13893a&scope.project.id=02308892635844b088ecdcb5413a21a0&role.id=b493ef4ee9d44f449e08251ae75a77d8 => generated 608 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:58:06.981248 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9732ff8-faf0-455d-88cc-ee2ef75dcb5e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:07.002067 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 316/630] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 12:58:06 2022] POST /identity/v3/endpoints => generated 346 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 12:58:07.114167 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9732ff8-faf0-455d-88cc-ee2ef75dcb5e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:07.125343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:07.177822 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c77d04b-a7bb-4653-88d3-a4f41c4dcd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:07.178385 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77d04b-a7bb-4653-88d3-a4f41c4dcd22 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:07.178663 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77d04b-a7bb-4653-88d3-a4f41c4dcd22 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:07.178931 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77d04b-a7bb-4653-88d3-a4f41c4dcd22 None admin] PATH_INFO: `/v3/services/network` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:07.179721 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c77d04b-a7bb-4653-88d3-a4f41c4dcd22 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:07.181112 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c77d04b-a7bb-4653-88d3-a4f41c4dcd22 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:07.195118 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-1c77d04b-a7bb-4653-88d3-a4f41c4dcd22 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 13 12:58:07.195118 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:07.195118 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 13 12:58:07.195118 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 13 12:58:07.195118 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 13 12:58:07.195118 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 13 12:58:07.195118 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 13 12:58:07.195118 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:07.195118 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 13 12:58:07.195118 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:07.195118 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 12:58:07.195118 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:07.195118 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:07.195118 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:07.195118 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:07.195118 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:07.195118 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:07.196184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:07.196184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:07.196184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:07.196184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:07.196184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:07.196184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:07.196184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 13 12:58:07.196184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 13 12:58:07.196184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 13 12:58:07.196184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 13 12:58:07.196184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:07.196184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:07.196184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:07.196184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:07.196184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:07.196184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:07.197022 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:07.197022 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:07.197022 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:07.197022 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:07.197022 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:07.197022 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:07.197022 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:07.197022 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:07.197022 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:07.197022 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:07.197022 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 13 12:58:07.197022 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:07.197022 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 13 12:58:07.197022 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:07.197022 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 315/631] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:58:06 2022] GET /identity/v3/services/network => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:07.202734 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44064d42-a9d1-4597-9a10-7b8405abe99e None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:07.327483 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44064d42-a9d1-4597-9a10-7b8405abe99e None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:07.353166 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:07.366771 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df275f-417f-4615-bdbf-a7986d75a421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:07.367482 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df275f-417f-4615-bdbf-a7986d75a421 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:07.367818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df275f-417f-4615-bdbf-a7986d75a421 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:07.368097 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df275f-417f-4615-bdbf-a7986d75a421 None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:07.368751 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df275f-417f-4615-bdbf-a7986d75a421 None admin] RBAC: Adding query filter params (name=network) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:07.369056 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df275f-417f-4615-bdbf-a7986d75a421 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:07.370505 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df275f-417f-4615-bdbf-a7986d75a421 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:07.377368 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 317/632] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Oct 13 12:58:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:07.383313 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c77d04b-a7bb-4653-88d3-a4f41c4dcd22 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:07.492792 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c77d04b-a7bb-4653-88d3-a4f41c4dcd22 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:07.527648 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1da13b1-f276-4604-ae35-6086174e9b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:07.528250 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da13b1-f276-4604-ae35-6086174e9b20 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:07.528527 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da13b1-f276-4604-ae35-6086174e9b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:07.528799 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da13b1-f276-4604-ae35-6086174e9b20 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:07.529427 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1da13b1-f276-4604-ae35-6086174e9b20 None admin] RBAC: Adding query filter params (type=network) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:07.529728 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1da13b1-f276-4604-ae35-6086174e9b20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:07.531111 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1da13b1-f276-4604-ae35-6086174e9b20 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:07.536354 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 316/633] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Oct 13 12:58:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:07.542500 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df275f-417f-4615-bdbf-a7986d75a421 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:07.662592 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df275f-417f-4615-bdbf-a7986d75a421 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:07.710571 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a4b38-0e87-4182-9588-cc7c4acdbc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:07.711620 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a4b38-0e87-4182-9588-cc7c4acdbc60 None admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:07.712122 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a4b38-0e87-4182-9588-cc7c4acdbc60 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:07.712592 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a4b38-0e87-4182-9588-cc7c4acdbc60 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:07.714069 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a4b38-0e87-4182-9588-cc7c4acdbc60 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:07.715668 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a4b38-0e87-4182-9588-cc7c4acdbc60 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:07.738813 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 318/634] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 12:58:07 2022] POST /identity/v3/endpoints => generated 337 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:58:07.875099 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91067d3-aad9-43b6-abb2-d60439fa0896 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:07.875390 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91067d3-aad9-43b6-abb2-d60439fa0896 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:07.875672 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91067d3-aad9-43b6-abb2-d60439fa0896 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:07.877287 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 317/635] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:07.882973 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880b353-d91c-4a7a-b679-cbd0cf92339a None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:07.883407 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880b353-d91c-4a7a-b679-cbd0cf92339a None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:07.883784 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880b353-d91c-4a7a-b679-cbd0cf92339a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:07.916038 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-7880b353-d91c-4a7a-b679-cbd0cf92339a None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:07.917274 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-7880b353-d91c-4a7a-b679-cbd0cf92339a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:07.980303 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:08.018071 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 319/636] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:07 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 13 12:58:08.294971 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91067d3-aad9-43b6-abb2-d60439fa0896 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:08.331474 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:08.422823 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91067d3-aad9-43b6-abb2-d60439fa0896 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:08.462169 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0535a71-ce99-49ec-ae2c-db12748b55c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:08.462792 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0535a71-ce99-49ec-ae2c-db12748b55c2 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:08.463105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0535a71-ce99-49ec-ae2c-db12748b55c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:08.463380 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0535a71-ce99-49ec-ae2c-db12748b55c2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:08.463956 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0535a71-ce99-49ec-ae2c-db12748b55c2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:08.465324 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0535a71-ce99-49ec-ae2c-db12748b55c2 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:08.474124 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-c0535a71-ce99-49ec-ae2c-db12748b55c2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 13 12:58:08.474124 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:08.474124 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 13 12:58:08.474124 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 13 12:58:08.474124 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 13 12:58:08.474124 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 13 12:58:08.474124 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 13 12:58:08.474124 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:08.474124 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 13 12:58:08.474124 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:08.474124 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 12:58:08.474124 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:08.474124 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:08.474124 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:08.474124 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:08.474124 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:08.474124 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:08.475072 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:08.475072 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:08.475072 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:08.475072 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:08.475072 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:08.475072 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:08.475072 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 13 12:58:08.475072 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 13 12:58:08.475072 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 13 12:58:08.475072 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 13 12:58:08.475072 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:08.475072 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:08.475072 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:08.475072 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:08.475072 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:08.475072 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:08.475926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:08.475926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:08.475926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:08.475926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:08.475926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:08.475926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:08.475926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:08.475926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:08.475926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:08.475926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:08.475926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 13 12:58:08.475926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:08.475926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 13 12:58:08.475926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:08.475926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 318/637] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Oct 13 12:58:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:08.481601 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880b353-d91c-4a7a-b679-cbd0cf92339a None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:08.593294 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880b353-d91c-4a7a-b679-cbd0cf92339a None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:08.631995 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd25f90f-5d8a-42aa-8673-9becec4acbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:08.632603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25f90f-5d8a-42aa-8673-9becec4acbaa None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:08.632881 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25f90f-5d8a-42aa-8673-9becec4acbaa None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:08.633149 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25f90f-5d8a-42aa-8673-9becec4acbaa None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:08.633780 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd25f90f-5d8a-42aa-8673-9becec4acbaa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:08.634082 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd25f90f-5d8a-42aa-8673-9becec4acbaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:08.635513 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd25f90f-5d8a-42aa-8673-9becec4acbaa None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:08.640305 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 320/638] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Oct 13 12:58:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:08.645585 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0535a71-ce99-49ec-ae2c-db12748b55c2 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:08.708588 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af52ace5-043f-4986-ab42-f00e3955262b None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:08.708945 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af52ace5-043f-4986-ab42-f00e3955262b None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:08.709156 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af52ace5-043f-4986-ab42-f00e3955262b None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:08.710634 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 321/639] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:08.715954 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0907fc-e0d1-4922-b277-33fcd7e17701 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:08.716478 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0907fc-e0d1-4922-b277-33fcd7e17701 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:08.716960 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0907fc-e0d1-4922-b277-33fcd7e17701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:08.736494 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-aa0907fc-e0d1-4922-b277-33fcd7e17701 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:08.738113 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-aa0907fc-e0d1-4922-b277-33fcd7e17701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:08.753497 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:08.761670 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0535a71-ce99-49ec-ae2c-db12748b55c2 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:08.768579 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:08.809161 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be32741-0e64-4bc2-9eda-5433e9ba9fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:08.809792 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be32741-0e64-4bc2-9eda-5433e9ba9fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:08.810071 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be32741-0e64-4bc2-9eda-5433e9ba9fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:08.810340 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be32741-0e64-4bc2-9eda-5433e9ba9fd7 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:08.811283 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be32741-0e64-4bc2-9eda-5433e9ba9fd7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:08.811593 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be32741-0e64-4bc2-9eda-5433e9ba9fd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:08.812970 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be32741-0e64-4bc2-9eda-5433e9ba9fd7 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:08.817660 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 319/640] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Oct 13 12:58:08 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:08.823246 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 322/641] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:08 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 13 12:58:09.096683 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be32741-0e64-4bc2-9eda-5433e9ba9fd7 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:09.201701 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be32741-0e64-4bc2-9eda-5433e9ba9fd7 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:09.236664 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab6d3eb-7f4e-49ef-a819-fa38bd0522d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:09.237645 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6d3eb-7f4e-49ef-a819-fa38bd0522d3 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:09.238118 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6d3eb-7f4e-49ef-a819-fa38bd0522d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:09.238665 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6d3eb-7f4e-49ef-a819-fa38bd0522d3 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:09.239541 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab6d3eb-7f4e-49ef-a819-fa38bd0522d3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:09.241112 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab6d3eb-7f4e-49ef-a819-fa38bd0522d3 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:09.263206 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 320/642] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Oct 13 12:58:09 2022] POST /identity/v3/services => generated 269 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:58:09.689032 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b494532-6c56-47cc-a4e7-6d3e9ee9444a None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:09.689559 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b494532-6c56-47cc-a4e7-6d3e9ee9444a None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:09.689559 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b494532-6c56-47cc-a4e7-6d3e9ee9444a None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:09.691223 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 323/643] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:09.697161 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ffa90-2f49-48ab-bbd9-137e5ac3bfd4 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:09.698298 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ffa90-2f49-48ab-bbd9-137e5ac3bfd4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:09.698298 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36ffa90-2f49-48ab-bbd9-137e5ac3bfd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:09.698573 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba62a4-8c3b-47a8-b620-bead73089853 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:09.698820 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba62a4-8c3b-47a8-b620-bead73089853 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:09.699120 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba62a4-8c3b-47a8-b620-bead73089853 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:09.700798 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 324/644] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 13 12:58:09.706742 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a55e66-8a70-4ffc-be4b-43cf2b8433f8 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:09.706742 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a55e66-8a70-4ffc-be4b-43cf2b8433f8 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:09.707022 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a55e66-8a70-4ffc-be4b-43cf2b8433f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:09.718154 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c36ffa90-2f49-48ab-bbd9-137e5ac3bfd4 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:09.719525 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c36ffa90-2f49-48ab-bbd9-137e5ac3bfd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:09.726214 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-46a55e66-8a70-4ffc-be4b-43cf2b8433f8 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:09.727820 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-46a55e66-8a70-4ffc-be4b-43cf2b8433f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:09.734120 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:09.800620 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 321/645] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:09 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:58:09.814004 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 325/646] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:09 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 13 12:58:10.079833 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ffa90-2f49-48ab-bbd9-137e5ac3bfd4 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:10.084284 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a55e66-8a70-4ffc-be4b-43cf2b8433f8 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:10.191111 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36ffa90-2f49-48ab-bbd9-137e5ac3bfd4 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:10.204217 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:10.212359 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a55e66-8a70-4ffc-be4b-43cf2b8433f8 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:10.232882 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1a748-7203-4f04-a8ab-8dd69827b2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:10.233607 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1a748-7203-4f04-a8ab-8dd69827b2c9 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:10.233919 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1a748-7203-4f04-a8ab-8dd69827b2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:10.234254 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1a748-7203-4f04-a8ab-8dd69827b2c9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:10.245429 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1a748-7203-4f04-a8ab-8dd69827b2c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:58:10.250475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-9de1a748-7203-4f04-a8ab-8dd69827b2c9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 13 12:58:10.250475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:10.250475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:10.250475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:10.250475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:10.250475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:10.250475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:10.250475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:10.250475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:10.250475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:10.250475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:10.250475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:10.250475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:58:10.250475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:58:10.250475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:58:10.250475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:58:10.251475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:58:10.251475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:58:10.251475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:58:10.251475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:58:10.251475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:10.251475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:10.251475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:10.251475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:10.251475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:10.251475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:10.251475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:10.251475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:10.251475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:10.251475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:10.251475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:10.251475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:10.252513 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:10.252513 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:10.252513 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:10.252513 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:10.252513 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:58:10.252513 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:58:10.252513 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:58:10.252513 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:58:10.252513 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:58:10.252513 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:58:10.252513 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 13 12:58:10.252513 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:10.256413 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 322/647] 10.222.0.23 () {56 vars in 1120 bytes} [Thu Oct 13 12:58:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:10.257000 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73b2f2-d54c-480b-8b74-324be1dd0113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:10.257320 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1a748-7203-4f04-a8ab-8dd69827b2c9 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:10.257860 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73b2f2-d54c-480b-8b74-324be1dd0113 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:10.258203 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73b2f2-d54c-480b-8b74-324be1dd0113 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:10.258541 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73b2f2-d54c-480b-8b74-324be1dd0113 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:10.259212 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf73b2f2-d54c-480b-8b74-324be1dd0113 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:10.260974 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf73b2f2-d54c-480b-8b74-324be1dd0113 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:10.266865 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:10.274430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-cf73b2f2-d54c-480b-8b74-324be1dd0113 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 13 12:58:10.274430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:10.274430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 13 12:58:10.274430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 13 12:58:10.274430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 13 12:58:10.274430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 13 12:58:10.274430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 13 12:58:10.274430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:10.274430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 13 12:58:10.274430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:10.274430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 12:58:10.274430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:10.274430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:10.274430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:10.274430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:10.274430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:10.274430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:10.276362 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:10.276362 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:10.276362 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:10.276362 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:10.276362 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:10.276362 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:10.276362 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 13 12:58:10.276362 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 13 12:58:10.276362 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 13 12:58:10.276362 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 13 12:58:10.276362 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:10.276362 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:10.276362 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:10.276362 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:10.276362 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:10.276362 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:10.277556 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:10.277556 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:10.277556 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:10.277556 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:10.277556 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:10.277556 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:10.277556 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:10.277556 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:10.277556 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:10.277556 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:10.277556 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 13 12:58:10.277556 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:10.277556 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 13 12:58:10.277556 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:10.280831 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 326/648] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Oct 13 12:58:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:10.282058 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73b2f2-d54c-480b-8b74-324be1dd0113 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:10.386205 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1a748-7203-4f04-a8ab-8dd69827b2c9 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:10.397773 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73b2f2-d54c-480b-8b74-324be1dd0113 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:10.421075 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d1c50-f692-4e8c-8d8c-be441b7473b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:10.421779 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d1c50-f692-4e8c-8d8c-be441b7473b3 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:10.422126 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d1c50-f692-4e8c-8d8c-be441b7473b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:10.422467 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d1c50-f692-4e8c-8d8c-be441b7473b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:10.423232 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d1c50-f692-4e8c-8d8c-be441b7473b3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:10.423603 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d1c50-f692-4e8c-8d8c-be441b7473b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:10.425239 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d1c50-f692-4e8c-8d8c-be441b7473b3 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:10.438882 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 323/649] 10.222.0.23 () {56 vars in 1115 bytes} [Thu Oct 13 12:58:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:58:10.439878 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d1c50-f692-4e8c-8d8c-be441b7473b3 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:10.442116 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c174479-ad5f-4002-b364-0c0415a39fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:10.443216 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c174479-ad5f-4002-b364-0c0415a39fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:10.444532 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c174479-ad5f-4002-b364-0c0415a39fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:10.444982 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c174479-ad5f-4002-b364-0c0415a39fd0 None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:10.445802 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c174479-ad5f-4002-b364-0c0415a39fd0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:10.446277 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c174479-ad5f-4002-b364-0c0415a39fd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:10.448480 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c174479-ad5f-4002-b364-0c0415a39fd0 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:10.460000 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 327/650] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Oct 13 12:58:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:10.461587 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c174479-ad5f-4002-b364-0c0415a39fd0 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:10.494697 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:10.514933 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:10.552588 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d1c50-f692-4e8c-8d8c-be441b7473b3 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:10.589918 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c174479-ad5f-4002-b364-0c0415a39fd0 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:10.590917 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b27935e-4657-4924-9874-8b8c0588db58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:10.591651 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27935e-4657-4924-9874-8b8c0588db58 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:10.592026 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27935e-4657-4924-9874-8b8c0588db58 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:10.592533 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27935e-4657-4924-9874-8b8c0588db58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:10.604503 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b27935e-4657-4924-9874-8b8c0588db58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:10.606164 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b27935e-4657-4924-9874-8b8c0588db58 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:10.622753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-4b27935e-4657-4924-9874-8b8c0588db58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:58:10.622753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:10.622753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:10.622753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:10.622753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:10.622753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:10.622753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:10.622753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:10.622753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:10.622753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:10.622753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:10.622753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:10.622753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:58:10.622753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:58:10.622753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:58:10.622753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:58:10.623776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:10.623776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:10.623776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:10.623776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:10.623776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:10.623776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:10.623776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:10.623776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:10.623776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:10.623776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:10.623776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:10.623776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:10.623776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:10.623776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:10.623776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:10.623776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:10.623776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:58:10.624665 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:58:10.624665 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:58:10.624665 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:10.628824 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 324/651] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:58:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:10.629752 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b27935e-4657-4924-9874-8b8c0588db58 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:10.632740 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1eac9-1065-432e-9946-651dbd338700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:10.634366 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1eac9-1065-432e-9946-651dbd338700 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:10.634963 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1eac9-1065-432e-9946-651dbd338700 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:10.635961 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1eac9-1065-432e-9946-651dbd338700 None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:10.636815 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1eac9-1065-432e-9946-651dbd338700 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:10.637290 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1eac9-1065-432e-9946-651dbd338700 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:10.638814 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1eac9-1065-432e-9946-651dbd338700 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:10.651256 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 328/652] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Oct 13 12:58:10 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:10.747937 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b27935e-4657-4924-9874-8b8c0588db58 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:10.769276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:10.785821 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cccd9f2-d76b-4f5b-bc01-c53c091a1c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:10.786428 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cccd9f2-d76b-4f5b-bc01-c53c091a1c95 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:10.786767 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cccd9f2-d76b-4f5b-bc01-c53c091a1c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:10.787105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cccd9f2-d76b-4f5b-bc01-c53c091a1c95 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:10.787759 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cccd9f2-d76b-4f5b-bc01-c53c091a1c95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:10.788059 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cccd9f2-d76b-4f5b-bc01-c53c091a1c95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:10.789439 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cccd9f2-d76b-4f5b-bc01-c53c091a1c95 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:10.803242 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 325/653] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:58:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:10.809605 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1eac9-1065-432e-9946-651dbd338700 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:10.906314 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:10.923436 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1eac9-1065-432e-9946-651dbd338700 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:10.957688 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78134c7c-ec78-4130-8b50-c51edefc4181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:10.958317 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78134c7c-ec78-4130-8b50-c51edefc4181 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:10.958593 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78134c7c-ec78-4130-8b50-c51edefc4181 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:10.958867 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78134c7c-ec78-4130-8b50-c51edefc4181 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:10.966807 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78134c7c-ec78-4130-8b50-c51edefc4181 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:10.968281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78134c7c-ec78-4130-8b50-c51edefc4181 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:10.979905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-78134c7c-ec78-4130-8b50-c51edefc4181 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 13 12:58:10.979905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:10.979905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:10.979905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:10.979905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:10.979905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:10.979905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:10.979905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:10.979905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:10.979905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:10.979905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:10.979905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:10.979905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 13 12:58:10.979905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 13 12:58:10.979905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 13 12:58:10.979905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 13 12:58:10.980851 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:10.980851 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:10.980851 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 13 12:58:10.980851 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:58:10.980851 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 13 12:58:10.980851 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:58:10.980851 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:10.980851 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:10.980851 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:10.980851 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:10.980851 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:10.980851 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:10.980851 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:10.980851 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:10.980851 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:10.980851 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:10.980851 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:10.980851 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:10.981782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:10.981782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:10.981782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 13 12:58:10.981782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 13 12:58:10.981782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 13 12:58:10.981782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 13 12:58:10.981782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 13 12:58:10.981782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 13 12:58:10.981782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 13 12:58:10.981782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:10.981782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 329/654] 10.222.0.23 () {56 vars in 1117 bytes} [Thu Oct 13 12:58:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:10.988614 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cccd9f2-d76b-4f5b-bc01-c53c091a1c95 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:10.990439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f58a1a-3597-4dea-bb6a-87a70269df23 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:10.990726 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f58a1a-3597-4dea-bb6a-87a70269df23 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:10.991000 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f58a1a-3597-4dea-bb6a-87a70269df23 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:10.992807 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 330/655] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 13 12:58:10.998441 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59442500-373c-4276-b74b-dbdf3745c1eb None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:10.998726 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59442500-373c-4276-b74b-dbdf3745c1eb None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:10.998999 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59442500-373c-4276-b74b-dbdf3745c1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:11.017894 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-59442500-373c-4276-b74b-dbdf3745c1eb None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:11.019092 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-59442500-373c-4276-b74b-dbdf3745c1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:11.087822 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 331/656] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:10 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:58:11.101058 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cccd9f2-d76b-4f5b-bc01-c53c091a1c95 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:11.138077 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b4494-b0be-4464-941c-dec131f994a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:11.138690 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b4494-b0be-4464-941c-dec131f994a8 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:11.138971 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b4494-b0be-4464-941c-dec131f994a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:11.139273 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b4494-b0be-4464-941c-dec131f994a8 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:11.139996 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b4494-b0be-4464-941c-dec131f994a8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:11.140299 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b4494-b0be-4464-941c-dec131f994a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:11.141784 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b4494-b0be-4464-941c-dec131f994a8 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:11.159873 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 326/657] 10.222.0.23 () {56 vars in 1149 bytes} [Thu Oct 13 12:58:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:11.165511 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59442500-373c-4276-b74b-dbdf3745c1eb None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:11.244663 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:11.273700 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59442500-373c-4276-b74b-dbdf3745c1eb None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:11.308684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d662a3b2-04e6-484b-a246-49d3397c010c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:11.309289 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662a3b2-04e6-484b-a246-49d3397c010c None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:11.309565 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662a3b2-04e6-484b-a246-49d3397c010c None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:11.309832 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662a3b2-04e6-484b-a246-49d3397c010c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:11.320502 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d662a3b2-04e6-484b-a246-49d3397c010c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:11.321944 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d662a3b2-04e6-484b-a246-49d3397c010c None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:11.336928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-d662a3b2-04e6-484b-a246-49d3397c010c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:58:11.336928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:11.336928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:11.336928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:11.336928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:11.336928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:11.336928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:11.336928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:11.336928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:11.336928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:11.336928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:11.336928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:11.336928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:58:11.336928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:58:11.336928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:58:11.336928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:58:11.338849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:11.338849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:11.338849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:11.338849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:11.338849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:11.338849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:11.338849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:11.338849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:11.338849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:11.338849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:11.338849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:11.338849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:11.338849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:11.338849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:11.338849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:11.338849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:11.338849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:58:11.339873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:58:11.339873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:58:11.339873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:11.339873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 332/658] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:58:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:11.343841 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b4494-b0be-4464-941c-dec131f994a8 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:11.373996 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d662a3b2-04e6-484b-a246-49d3397c010c None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:11.389951 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:11.455654 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b4494-b0be-4464-941c-dec131f994a8 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:11.483311 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d662a3b2-04e6-484b-a246-49d3397c010c None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:11.493971 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281bf438-3fdb-4ae5-af9f-fa95771c5020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:11.495405 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281bf438-3fdb-4ae5-af9f-fa95771c5020 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:11.495870 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281bf438-3fdb-4ae5-af9f-fa95771c5020 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:11.496398 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281bf438-3fdb-4ae5-af9f-fa95771c5020 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:11.497072 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281bf438-3fdb-4ae5-af9f-fa95771c5020 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:11.497378 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281bf438-3fdb-4ae5-af9f-fa95771c5020 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:11.498736 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281bf438-3fdb-4ae5-af9f-fa95771c5020 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:11.510425 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 327/659] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:58:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:11.511479 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17325b-e9f9-4894-85b8-f8c83fa9e120 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:11.511763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17325b-e9f9-4894-85b8-f8c83fa9e120 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:11.512101 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17325b-e9f9-4894-85b8-f8c83fa9e120 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:11.512597 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:11.513435 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 328/660] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:11.518942 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae0e77-66e3-40c4-977b-4b5415d1a4ad None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:11.518942 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae0e77-66e3-40c4-977b-4b5415d1a4ad None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:11.519350 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae0e77-66e3-40c4-977b-4b5415d1a4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:11.520054 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b83f5a-ff18-4a73-bfe5-29ee23680abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:11.520909 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b83f5a-ff18-4a73-bfe5-29ee23680abb None admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:11.521079 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b83f5a-ff18-4a73-bfe5-29ee23680abb None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:11.521347 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b83f5a-ff18-4a73-bfe5-29ee23680abb None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:11.522511 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b83f5a-ff18-4a73-bfe5-29ee23680abb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:11.524087 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b83f5a-ff18-4a73-bfe5-29ee23680abb None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:11.537671 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c9ae0e77-66e3-40c4-977b-4b5415d1a4ad None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:11.538854 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c9ae0e77-66e3-40c4-977b-4b5415d1a4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:11.543229 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 333/661] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Oct 13 12:58:11 2022] POST /identity/v3/services => generated 241 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 12:58:11.544168 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b83f5a-ff18-4a73-bfe5-29ee23680abb None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:11.641528 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 329/662] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:11 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 13 12:58:11.679640 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b83f5a-ff18-4a73-bfe5-29ee23680abb None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:11.718117 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378cb8b3-d1ca-43e3-942b-14cfe58e5619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:11.718678 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378cb8b3-d1ca-43e3-942b-14cfe58e5619 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:11.720411 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378cb8b3-d1ca-43e3-942b-14cfe58e5619 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:11.720411 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378cb8b3-d1ca-43e3-942b-14cfe58e5619 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:11.726631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378cb8b3-d1ca-43e3-942b-14cfe58e5619 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:11.728096 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378cb8b3-d1ca-43e3-942b-14cfe58e5619 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:11.740872 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-378cb8b3-d1ca-43e3-942b-14cfe58e5619 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:58:11.740872 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:11.740872 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:11.740872 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:11.740872 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:11.740872 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:11.740872 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:11.740872 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:11.740872 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:11.740872 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:11.740872 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:11.740872 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:11.740872 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 13 12:58:11.740872 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 13 12:58:11.740872 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 13 12:58:11.741791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 13 12:58:11.741791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:11.741791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:11.741791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:11.741791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:11.741791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:11.741791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:11.741791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:11.741791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:11.741791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:11.741791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:11.741791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:11.741791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:11.741791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:11.741791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:11.741791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:11.741791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:11.743324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 13 12:58:11.743324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 13 12:58:11.743324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 13 12:58:11.743324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 13 12:58:11.743324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 13 12:58:11.743324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 13 12:58:11.743324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:58:11.743324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:11.743324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 334/663] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:58:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:11.750723 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae0e77-66e3-40c4-977b-4b5415d1a4ad None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:11.775536 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:11.851535 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae0e77-66e3-40c4-977b-4b5415d1a4ad None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:11.886921 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74427f15-c7cc-4af0-a791-e2d60021ea77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:11.887551 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74427f15-c7cc-4af0-a791-e2d60021ea77 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:11.887826 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74427f15-c7cc-4af0-a791-e2d60021ea77 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:11.888095 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74427f15-c7cc-4af0-a791-e2d60021ea77 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:11.888776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74427f15-c7cc-4af0-a791-e2d60021ea77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:11.889074 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74427f15-c7cc-4af0-a791-e2d60021ea77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:11.890486 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74427f15-c7cc-4af0-a791-e2d60021ea77 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:11.902159 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 330/664] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 13 12:58:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:58:11.908085 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378cb8b3-d1ca-43e3-942b-14cfe58e5619 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:11.918023 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74427f15-c7cc-4af0-a791-e2d60021ea77 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:11.969369 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:12.021476 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:12.035236 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378cb8b3-d1ca-43e3-942b-14cfe58e5619 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:12.039082 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74427f15-c7cc-4af0-a791-e2d60021ea77 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:12.077195 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ef262-cbcd-425f-adfc-17fc982fe684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:12.077824 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ef262-cbcd-425f-adfc-17fc982fe684 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:12.078099 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ef262-cbcd-425f-adfc-17fc982fe684 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:12.078371 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ef262-cbcd-425f-adfc-17fc982fe684 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:12.079036 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ef262-cbcd-425f-adfc-17fc982fe684 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:12.080429 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ef262-cbcd-425f-adfc-17fc982fe684 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:12.082603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d2abc8-1b93-4bc3-86a3-14ed707c78a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:12.083980 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2abc8-1b93-4bc3-86a3-14ed707c78a8 None admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:12.084659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2abc8-1b93-4bc3-86a3-14ed707c78a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:12.085078 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2abc8-1b93-4bc3-86a3-14ed707c78a8 None admin] PATH_INFO: `/v3/projects/02308892635844b088ecdcb5413a21a0/users/b6ae674f0efc445fa970fca64e13893a/roles/13498f16883f4cf0b7d010ec7e4d0866` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:12.087428 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d2abc8-1b93-4bc3-86a3-14ed707c78a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=02308892635844b088ecdcb5413a21a0, user_id=b6ae674f0efc445fa970fca64e13893a, role_id=13498f16883f4cf0b7d010ec7e4d0866)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:12.089039 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d2abc8-1b93-4bc3-86a3-14ed707c78a8 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:12.089943 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-f87ef262-cbcd-425f-adfc-17fc982fe684 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 13 12:58:12.089943 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:12.089943 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 13 12:58:12.089943 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 13 12:58:12.089943 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 13 12:58:12.089943 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 13 12:58:12.089943 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 13 12:58:12.089943 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:12.089943 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 13 12:58:12.089943 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:12.089943 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 12:58:12.089943 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:12.089943 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:12.089943 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:12.089943 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:12.089943 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:12.091076 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:12.091076 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:12.091076 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:12.091076 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:12.091076 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:12.091076 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:12.091076 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:12.091076 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 13 12:58:12.091076 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 13 12:58:12.091076 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 13 12:58:12.091076 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 13 12:58:12.091076 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:12.091076 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:12.091076 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:12.091076 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:12.091076 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:12.091076 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:12.092223 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:12.092223 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:12.092223 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:12.092223 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:12.092223 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:12.092223 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:12.092223 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:12.092223 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:12.092223 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:12.092223 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:12.092223 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 13 12:58:12.092223 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:12.092223 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 13 12:58:12.092223 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:12.098720 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 331/665] 10.222.0.23 () {56 vars in 1132 bytes} [Thu Oct 13 12:58:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:58:12.099785 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ef262-cbcd-425f-adfc-17fc982fe684 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:12.113098 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 335/666] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 13 12:58:11 2022] PUT /identity/v3/projects/02308892635844b088ecdcb5413a21a0/users/b6ae674f0efc445fa970fca64e13893a/roles/13498f16883f4cf0b7d010ec7e4d0866 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 12:58:12.223792 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ef262-cbcd-425f-adfc-17fc982fe684 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:12.262945 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e12a7-bde5-48c0-925a-e4456b6b5fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:12.263875 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e12a7-bde5-48c0-925a-e4456b6b5fbd None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:12.264299 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e12a7-bde5-48c0-925a-e4456b6b5fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:12.264716 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e12a7-bde5-48c0-925a-e4456b6b5fbd None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:12.265502 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e12a7-bde5-48c0-925a-e4456b6b5fbd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:12.265999 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e12a7-bde5-48c0-925a-e4456b6b5fbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:12.267620 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e12a7-bde5-48c0-925a-e4456b6b5fbd None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:12.273665 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 332/667] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Oct 13 12:58:12 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:12.279691 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d2abc8-1b93-4bc3-86a3-14ed707c78a8 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:12.386650 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d2abc8-1b93-4bc3-86a3-14ed707c78a8 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:12.396310 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:12.424853 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde11d58-9a0b-4100-bf88-f4cb26c2a1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:12.425504 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde11d58-9a0b-4100-bf88-f4cb26c2a1cb None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:12.425784 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde11d58-9a0b-4100-bf88-f4cb26c2a1cb None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:12.426051 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde11d58-9a0b-4100-bf88-f4cb26c2a1cb None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:12.426737 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde11d58-9a0b-4100-bf88-f4cb26c2a1cb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:12.427039 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde11d58-9a0b-4100-bf88-f4cb26c2a1cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:12.428420 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde11d58-9a0b-4100-bf88-f4cb26c2a1cb None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:12.433848 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 336/668] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Oct 13 12:58:12 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:12.439985 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e12a7-bde5-48c0-925a-e4456b6b5fbd None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:12.456184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:12.550346 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e12a7-bde5-48c0-925a-e4456b6b5fbd None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:12.586612 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d2ee7-97d6-40ea-ab88-fa53967a113a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:12.587250 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d2ee7-97d6-40ea-ab88-fa53967a113a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:12.587531 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d2ee7-97d6-40ea-ab88-fa53967a113a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:12.587845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d2ee7-97d6-40ea-ab88-fa53967a113a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:12.589059 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d2ee7-97d6-40ea-ab88-fa53967a113a None admin] RBAC: Adding query filter params (interface=public, service_id=bf20397ec3f04ccd854991565204ff6b, region_id=RegionOne) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:12.589366 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d2ee7-97d6-40ea-ab88-fa53967a113a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:12.590720 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d2ee7-97d6-40ea-ab88-fa53967a113a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:12.596439 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 333/669] 10.222.0.23 () {56 vars in 1251 bytes} [Thu Oct 13 12:58:12 2022] GET /identity/v3/endpoints?service_id=bf20397ec3f04ccd854991565204ff6b&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:12.601984 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde11d58-9a0b-4100-bf88-f4cb26c2a1cb None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:12.703852 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde11d58-9a0b-4100-bf88-f4cb26c2a1cb None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:12.737146 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdfdfbc-6b5e-4aaf-97f1-b6d2fe10c8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:12.737757 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdfdfbc-6b5e-4aaf-97f1-b6d2fe10c8dd None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:12.738037 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdfdfbc-6b5e-4aaf-97f1-b6d2fe10c8dd None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:12.738309 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdfdfbc-6b5e-4aaf-97f1-b6d2fe10c8dd None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:12.738914 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdfdfbc-6b5e-4aaf-97f1-b6d2fe10c8dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:12.740309 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdfdfbc-6b5e-4aaf-97f1-b6d2fe10c8dd None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:12.746981 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 337/670] 10.222.0.23 () {56 vars in 1087 bytes} [Thu Oct 13 12:58:12 2022] GET /identity/v3/services => generated 1258 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 12:58:12.930626 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79f367-f371-4d76-99fd-616e4811e2d0 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:12.931284 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79f367-f371-4d76-99fd-616e4811e2d0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:12.931284 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79f367-f371-4d76-99fd-616e4811e2d0 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:12.933363 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 334/671] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:12.938971 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30da56-2bd3-42db-9204-507da4a79907 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:12.939437 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30da56-2bd3-42db-9204-507da4a79907 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:12.939631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30da56-2bd3-42db-9204-507da4a79907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:12.960034 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-bf30da56-2bd3-42db-9204-507da4a79907 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:12.961198 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-bf30da56-2bd3-42db-9204-507da4a79907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:12.990956 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:13.030912 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 338/672] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:12 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:58:13.303279 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79f367-f371-4d76-99fd-616e4811e2d0 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:13.383944 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:13.412905 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79f367-f371-4d76-99fd-616e4811e2d0 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:13.448701 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdcef2f-19e5-4954-9784-6207a99729c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:13.449311 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcef2f-19e5-4954-9784-6207a99729c6 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:13.449589 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcef2f-19e5-4954-9784-6207a99729c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:13.449858 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcef2f-19e5-4954-9784-6207a99729c6 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:13.450531 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdcef2f-19e5-4954-9784-6207a99729c6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:13.451914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdcef2f-19e5-4954-9784-6207a99729c6 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:13.469461 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 335/673] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Oct 13 12:58:13 2022] POST /identity/v3/services => generated 251 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:58:13.823920 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b2e11-8f41-475f-a5ce-22eb31a00178 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:13.824301 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b2e11-8f41-475f-a5ce-22eb31a00178 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:13.824501 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b2e11-8f41-475f-a5ce-22eb31a00178 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:13.826051 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 339/674] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:13.831500 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627d4f6-52f9-4dee-9812-9ff3a92d1fd4 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:13.831958 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627d4f6-52f9-4dee-9812-9ff3a92d1fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:13.832238 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627d4f6-52f9-4dee-9812-9ff3a92d1fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:13.850857 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6627d4f6-52f9-4dee-9812-9ff3a92d1fd4 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:13.852057 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6627d4f6-52f9-4dee-9812-9ff3a92d1fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:13.937811 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 336/675] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:58:14.206283 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b2e11-8f41-475f-a5ce-22eb31a00178 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:14.314158 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b2e11-8f41-475f-a5ce-22eb31a00178 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:14.354861 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034225ab-57a4-4e9c-9c3f-84f417d28307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:14.355928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034225ab-57a4-4e9c-9c3f-84f417d28307 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:14.356404 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034225ab-57a4-4e9c-9c3f-84f417d28307 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:14.356870 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034225ab-57a4-4e9c-9c3f-84f417d28307 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:14.357719 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034225ab-57a4-4e9c-9c3f-84f417d28307 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:14.359550 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034225ab-57a4-4e9c-9c3f-84f417d28307 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:14.370472 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-034225ab-57a4-4e9c-9c3f-84f417d28307 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 13 12:58:14.370472 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:14.370472 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 13 12:58:14.370472 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 13 12:58:14.370472 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 13 12:58:14.370472 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 13 12:58:14.370472 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 13 12:58:14.370472 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:14.370472 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 13 12:58:14.370472 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:14.370472 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 12:58:14.370472 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:14.370472 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:14.370472 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:14.370472 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:14.370472 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:14.370472 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:14.371603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:14.371603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:14.371603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:14.371603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:14.371603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:14.371603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:14.371603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 13 12:58:14.371603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 13 12:58:14.371603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 13 12:58:14.371603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 13 12:58:14.371603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:14.371603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:14.371603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:14.371603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:14.371603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:14.371603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:14.372486 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:14.372486 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:14.372486 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:14.372486 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:14.372486 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:14.372486 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:14.372486 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:14.372486 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:14.372486 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:14.372486 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:14.372486 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 13 12:58:14.372486 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:14.372486 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 13 12:58:14.372486 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:14.375355 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 340/676] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Oct 13 12:58:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:14.387102 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6627d4f6-52f9-4dee-9812-9ff3a92d1fd4 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:14.442500 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:14.457965 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475aa10c-af88-428d-af37-899d827efa74 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:14.458241 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475aa10c-af88-428d-af37-899d827efa74 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:14.458438 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475aa10c-af88-428d-af37-899d827efa74 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:14.460259 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 341/677] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:14.467268 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec2453-4878-4799-8c2e-61ab67fbbf6e None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:14.467544 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec2453-4878-4799-8c2e-61ab67fbbf6e None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:14.467811 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec2453-4878-4799-8c2e-61ab67fbbf6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:14.488437 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-bdec2453-4878-4799-8c2e-61ab67fbbf6e None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:14.489731 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-bdec2453-4878-4799-8c2e-61ab67fbbf6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:14.490384 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6627d4f6-52f9-4dee-9812-9ff3a92d1fd4 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:14.519674 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:14.536378 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb01c7b-21ac-421d-af03-fe0e17964bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:14.536989 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb01c7b-21ac-421d-af03-fe0e17964bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:14.537266 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb01c7b-21ac-421d-af03-fe0e17964bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:14.537532 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb01c7b-21ac-421d-af03-fe0e17964bd1 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:14.538154 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb01c7b-21ac-421d-af03-fe0e17964bd1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:14.538451 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb01c7b-21ac-421d-af03-fe0e17964bd1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:14.540010 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb01c7b-21ac-421d-af03-fe0e17964bd1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:14.547651 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 337/678] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Oct 13 12:58:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:14.553243 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb01c7b-21ac-421d-af03-fe0e17964bd1 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:14.575391 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 342/679] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 13 12:58:14.661030 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb01c7b-21ac-421d-af03-fe0e17964bd1 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:14.687590 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:14.696761 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03ff94-f8b9-41c1-875b-0ce6821e1aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:14.697435 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03ff94-f8b9-41c1-875b-0ce6821e1aba None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:14.697858 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03ff94-f8b9-41c1-875b-0ce6821e1aba None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:14.698182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03ff94-f8b9-41c1-875b-0ce6821e1aba None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:14.698997 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed03ff94-f8b9-41c1-875b-0ce6821e1aba None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:14.699326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed03ff94-f8b9-41c1-875b-0ce6821e1aba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:14.700678 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed03ff94-f8b9-41c1-875b-0ce6821e1aba None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:14.709452 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 338/680] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Oct 13 12:58:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:58:14.712124 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdec2453-4878-4799-8c2e-61ab67fbbf6e None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:14.816128 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdec2453-4878-4799-8c2e-61ab67fbbf6e None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:14.842023 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03ff94-f8b9-41c1-875b-0ce6821e1aba None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:14.852569 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46931a39-4abe-4be3-a7f7-b962c546178f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:14.854117 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46931a39-4abe-4be3-a7f7-b962c546178f None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:14.854406 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46931a39-4abe-4be3-a7f7-b962c546178f None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:14.854683 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46931a39-4abe-4be3-a7f7-b962c546178f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:14.855723 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46931a39-4abe-4be3-a7f7-b962c546178f None admin] RBAC: Adding query filter params (interface=public, service_id=1ef6ce68fefb4a0598606730f59beecd, region_id=RegionOne) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:14.856169 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46931a39-4abe-4be3-a7f7-b962c546178f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:14.857938 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46931a39-4abe-4be3-a7f7-b962c546178f None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:14.867167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 343/681] 10.222.0.23 () {56 vars in 1251 bytes} [Thu Oct 13 12:58:14 2022] GET /identity/v3/endpoints?service_id=1ef6ce68fefb4a0598606730f59beecd&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:14.870927 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46931a39-4abe-4be3-a7f7-b962c546178f None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:14.914272 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:14.969545 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03ff94-f8b9-41c1-875b-0ce6821e1aba None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:15.001900 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46931a39-4abe-4be3-a7f7-b962c546178f None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:15.013798 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf46c3-4adb-48fa-87be-c33615c553fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:15.014509 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf46c3-4adb-48fa-87be-c33615c553fa None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:15.014798 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf46c3-4adb-48fa-87be-c33615c553fa None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:15.015087 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf46c3-4adb-48fa-87be-c33615c553fa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:15.026435 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf46c3-4adb-48fa-87be-c33615c553fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:58:15.033656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-2abf46c3-4adb-48fa-87be-c33615c553fa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 13 12:58:15.033656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:15.033656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:15.033656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:15.033656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:15.033656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:15.033656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:15.033656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:15.033656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:15.033656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:15.033656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:15.033656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:15.033656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:58:15.033656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:58:15.033656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:58:15.033656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:58:15.034613 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:58:15.034613 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:58:15.034613 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:58:15.034613 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:58:15.034613 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:15.034613 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:15.034613 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:15.034613 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:15.034613 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:15.034613 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:15.034613 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:15.034613 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:15.034613 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:15.034613 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:15.034613 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:15.034613 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:15.035458 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:15.035458 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:15.035458 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:15.035458 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:15.035458 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:58:15.035458 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:58:15.035458 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:58:15.035458 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:58:15.035458 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:58:15.035458 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:58:15.035458 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 13 12:58:15.035458 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:15.036097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 339/682] 10.222.0.23 () {56 vars in 1120 bytes} [Thu Oct 13 12:58:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:58:15.036818 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f590ab-e313-4f7b-ac96-9d01e1e58f33 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:15.037102 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f590ab-e313-4f7b-ac96-9d01e1e58f33 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:15.037376 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f590ab-e313-4f7b-ac96-9d01e1e58f33 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:15.038749 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 340/683] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:15.043641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f590ab-e313-4f7b-ac96-9d01e1e58f33 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:15.043761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8fd3ce-757a-4970-a95a-53665db4dd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:15.044442 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8fd3ce-757a-4970-a95a-53665db4dd5c None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:15.044783 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8fd3ce-757a-4970-a95a-53665db4dd5c None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:15.045879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8fd3ce-757a-4970-a95a-53665db4dd5c None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:15.046517 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8fd3ce-757a-4970-a95a-53665db4dd5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:15.047915 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8fd3ce-757a-4970-a95a-53665db4dd5c None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:15.059129 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 344/684] 10.222.0.23 () {56 vars in 1087 bytes} [Thu Oct 13 12:58:14 2022] GET /identity/v3/services => generated 1497 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 12:58:15.059129 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91680313-ffc0-4e66-bb34-4ea573776372 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:15.059129 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91680313-ffc0-4e66-bb34-4ea573776372 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:15.059129 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91680313-ffc0-4e66-bb34-4ea573776372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:15.076969 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-91680313-ffc0-4e66-bb34-4ea573776372 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:15.078297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-91680313-ffc0-4e66-bb34-4ea573776372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:15.100444 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:15.158788 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f590ab-e313-4f7b-ac96-9d01e1e58f33 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:15.179756 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 345/685] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:58:15.209059 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbe16b0-c244-4463-9d49-cdeee746f2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:15.209978 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe16b0-c244-4463-9d49-cdeee746f2b1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:15.210402 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe16b0-c244-4463-9d49-cdeee746f2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:15.210900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe16b0-c244-4463-9d49-cdeee746f2b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:15.211775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbe16b0-c244-4463-9d49-cdeee746f2b1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:15.212228 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbe16b0-c244-4463-9d49-cdeee746f2b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:15.214443 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbe16b0-c244-4463-9d49-cdeee746f2b1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:15.224722 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 341/686] 10.222.0.23 () {56 vars in 1115 bytes} [Thu Oct 13 12:58:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:58:15.230968 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91680313-ffc0-4e66-bb34-4ea573776372 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:15.251262 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:15.340287 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91680313-ffc0-4e66-bb34-4ea573776372 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:15.375630 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3201fa-285c-46e7-9a71-a45eb12fa195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:15.376507 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3201fa-285c-46e7-9a71-a45eb12fa195 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:15.376931 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3201fa-285c-46e7-9a71-a45eb12fa195 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:15.377340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3201fa-285c-46e7-9a71-a45eb12fa195 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:15.390628 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3201fa-285c-46e7-9a71-a45eb12fa195 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:15.392280 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3201fa-285c-46e7-9a71-a45eb12fa195 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:15.407649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-4c3201fa-285c-46e7-9a71-a45eb12fa195 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:58:15.407649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:15.407649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:15.407649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:15.407649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:15.407649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:15.407649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:15.407649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:15.407649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:15.407649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:15.407649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:15.407649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:15.407649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:58:15.407649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:58:15.407649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:58:15.407649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:58:15.408777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:15.408777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:15.408777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:15.408777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:15.408777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:15.408777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:15.408777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:15.408777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:15.408777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:15.408777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:15.408777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:15.408777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:15.408777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:15.408777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:15.408777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:15.408777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:15.408777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:58:15.409893 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:58:15.409893 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:58:15.409893 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:15.411596 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 346/687] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:58:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:15.417379 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbe16b0-c244-4463-9d49-cdeee746f2b1 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:15.447857 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3201fa-285c-46e7-9a71-a45eb12fa195 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:15.527346 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbe16b0-c244-4463-9d49-cdeee746f2b1 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:15.542559 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:15.552153 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:15.568737 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa55f0-2f8b-4df7-aa30-d327fdfffc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:15.569583 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa55f0-2f8b-4df7-aa30-d327fdfffc3b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:15.569963 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa55f0-2f8b-4df7-aa30-d327fdfffc3b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:15.570333 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa55f0-2f8b-4df7-aa30-d327fdfffc3b None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:15.571226 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfa55f0-2f8b-4df7-aa30-d327fdfffc3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:15.571752 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfa55f0-2f8b-4df7-aa30-d327fdfffc3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:15.573831 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfa55f0-2f8b-4df7-aa30-d327fdfffc3b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:15.591086 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 342/688] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:58:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:15.592777 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa55f0-2f8b-4df7-aa30-d327fdfffc3b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:15.597429 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3201fa-285c-46e7-9a71-a45eb12fa195 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:15.643243 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905dbe73-2125-4d73-aab0-6c5b02f2c144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:15.644629 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905dbe73-2125-4d73-aab0-6c5b02f2c144 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:15.644911 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905dbe73-2125-4d73-aab0-6c5b02f2c144 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:15.645185 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905dbe73-2125-4d73-aab0-6c5b02f2c144 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:15.645791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905dbe73-2125-4d73-aab0-6c5b02f2c144 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:15.647708 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905dbe73-2125-4d73-aab0-6c5b02f2c144 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:15.660235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-905dbe73-2125-4d73-aab0-6c5b02f2c144 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 13 12:58:15.660235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:15.660235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 13 12:58:15.660235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 13 12:58:15.660235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 13 12:58:15.660235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 13 12:58:15.660235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 13 12:58:15.660235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:15.660235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 13 12:58:15.660235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:15.660235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 12:58:15.660235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:15.660235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:15.660235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:15.660235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:15.660235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:15.661238 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:15.661238 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:15.661238 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:15.661238 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:15.661238 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:15.661238 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:15.661238 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:15.661238 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 13 12:58:15.661238 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 13 12:58:15.661238 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 13 12:58:15.661238 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 13 12:58:15.661238 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:15.661238 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:15.661238 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:15.661238 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:15.661238 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:15.661238 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:15.662157 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:15.662157 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:15.662157 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:15.662157 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:15.662157 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:15.662157 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:15.662157 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:15.662157 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:15.662157 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:15.662157 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:15.662157 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 13 12:58:15.662157 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:15.662157 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 13 12:58:15.662157 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:15.667212 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 347/689] 10.222.0.23 () {56 vars in 1132 bytes} [Thu Oct 13 12:58:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:58:15.669438 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905dbe73-2125-4d73-aab0-6c5b02f2c144 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:15.719488 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa55f0-2f8b-4df7-aa30-d327fdfffc3b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:15.756558 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596389f-1d89-496c-961a-a1e7f7d5af99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:15.757290 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596389f-1d89-496c-961a-a1e7f7d5af99 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:15.757603 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596389f-1d89-496c-961a-a1e7f7d5af99 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:15.757843 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596389f-1d89-496c-961a-a1e7f7d5af99 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:15.766038 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596389f-1d89-496c-961a-a1e7f7d5af99 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:15.767513 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596389f-1d89-496c-961a-a1e7f7d5af99 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:15.779305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-e596389f-1d89-496c-961a-a1e7f7d5af99 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 13 12:58:15.779305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:15.779305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:15.779305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:15.779305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:15.779305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:15.779305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:15.779305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:15.779305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:15.779305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:15.779305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:15.779305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:15.779305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 13 12:58:15.779305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 13 12:58:15.779305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 13 12:58:15.779305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 13 12:58:15.780251 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:15.780251 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:15.780251 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 13 12:58:15.780251 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:58:15.780251 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 13 12:58:15.780251 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:58:15.780251 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:15.780251 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:15.780251 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:15.780251 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:15.780251 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:15.780251 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:15.780251 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:15.780251 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:15.780251 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:15.780251 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:15.780251 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:15.780251 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:15.781181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:15.781181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:15.781181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 13 12:58:15.781181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 13 12:58:15.781181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 13 12:58:15.781181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 13 12:58:15.781181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 13 12:58:15.781181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 13 12:58:15.781181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 13 12:58:15.781181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:15.781181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 343/690] 10.222.0.23 () {56 vars in 1117 bytes} [Thu Oct 13 12:58:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:58:15.781756 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fee4fe-75f0-4c2e-9790-e79e18dec6fe None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:15.781823 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fee4fe-75f0-4c2e-9790-e79e18dec6fe None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:15.782471 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fee4fe-75f0-4c2e-9790-e79e18dec6fe None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:15.783468 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 344/691] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:15.787876 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fee4fe-75f0-4c2e-9790-e79e18dec6fe None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:15.793768 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905dbe73-2125-4d73-aab0-6c5b02f2c144 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:15.822605 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:15.829565 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f198cc4-b8c5-47dc-b23b-214b36a9c532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:15.830164 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f198cc4-b8c5-47dc-b23b-214b36a9c532 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:15.830441 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f198cc4-b8c5-47dc-b23b-214b36a9c532 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:15.830707 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f198cc4-b8c5-47dc-b23b-214b36a9c532 None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:15.831360 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f198cc4-b8c5-47dc-b23b-214b36a9c532 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:15.831660 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f198cc4-b8c5-47dc-b23b-214b36a9c532 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:15.833055 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f198cc4-b8c5-47dc-b23b-214b36a9c532 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:15.834132 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:15.839634 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 348/692] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Oct 13 12:58:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:15.841142 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df49ec-a3fb-4092-83ed-b42d45de41c0 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:15.841647 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df49ec-a3fb-4092-83ed-b42d45de41c0 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:15.842126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df49ec-a3fb-4092-83ed-b42d45de41c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:15.861754 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-41df49ec-a3fb-4092-83ed-b42d45de41c0 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:15.862939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-41df49ec-a3fb-4092-83ed-b42d45de41c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:15.904889 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fee4fe-75f0-4c2e-9790-e79e18dec6fe None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:15.938106 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 349/693] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:58:15.940162 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df49ec-a3fb-4092-83ed-b42d45de41c0 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:15.942394 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5fac2-d965-4207-9ec2-a02df4e2bf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:15.942996 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5fac2-d965-4207-9ec2-a02df4e2bf58 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:15.943307 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5fac2-d965-4207-9ec2-a02df4e2bf58 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:15.943578 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5fac2-d965-4207-9ec2-a02df4e2bf58 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:15.944301 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e5fac2-d965-4207-9ec2-a02df4e2bf58 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:15.944605 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e5fac2-d965-4207-9ec2-a02df4e2bf58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:15.946020 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e5fac2-d965-4207-9ec2-a02df4e2bf58 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:15.970948 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 345/694] 10.222.0.23 () {56 vars in 1149 bytes} [Thu Oct 13 12:58:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:15.972353 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5fac2-d965-4207-9ec2-a02df4e2bf58 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:16.075846 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:16.077445 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df49ec-a3fb-4092-83ed-b42d45de41c0 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:16.086460 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5fac2-d965-4207-9ec2-a02df4e2bf58 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:16.106685 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:16.116123 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65df26-a35a-465f-b568-41e1e5e1bb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:16.116965 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65df26-a35a-465f-b568-41e1e5e1bb09 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:16.117486 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65df26-a35a-465f-b568-41e1e5e1bb09 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:16.118119 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65df26-a35a-465f-b568-41e1e5e1bb09 None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:16.119620 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e65df26-a35a-465f-b568-41e1e5e1bb09 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:16.120158 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e65df26-a35a-465f-b568-41e1e5e1bb09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:16.121869 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035fa70-9dbe-4412-a2e1-fcf45893eaca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:16.122161 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e65df26-a35a-465f-b568-41e1e5e1bb09 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:16.122624 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035fa70-9dbe-4412-a2e1-fcf45893eaca None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:16.123014 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035fa70-9dbe-4412-a2e1-fcf45893eaca None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:16.123396 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035fa70-9dbe-4412-a2e1-fcf45893eaca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:16.128327 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 350/695] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Oct 13 12:58:15 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:16.136108 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65df26-a35a-465f-b568-41e1e5e1bb09 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:16.205598 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9035fa70-9dbe-4412-a2e1-fcf45893eaca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:16.207396 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9035fa70-9dbe-4412-a2e1-fcf45893eaca None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:16.252348 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65df26-a35a-465f-b568-41e1e5e1bb09 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:16.290298 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c9374f-e792-4ccd-b566-2ab796e217af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:16.291451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9374f-e792-4ccd-b566-2ab796e217af None admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:16.291733 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9374f-e792-4ccd-b566-2ab796e217af None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:16.292004 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9374f-e792-4ccd-b566-2ab796e217af None admin] PATH_INFO: `/v3/endpoints` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:16.292659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c9374f-e792-4ccd-b566-2ab796e217af None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:16.294012 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c9374f-e792-4ccd-b566-2ab796e217af None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:16.300557 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:16.323413 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 346/696] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 13 12:58:15 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 13 12:58:16.323621 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 351/697] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 12:58:16 2022] POST /identity/v3/endpoints => generated 347 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:58:16.323716 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035fa70-9dbe-4412-a2e1-fcf45893eaca None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:16.329352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c9374f-e792-4ccd-b566-2ab796e217af None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:16.385447 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:16.453119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035fa70-9dbe-4412-a2e1-fcf45893eaca None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:16.455716 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c9374f-e792-4ccd-b566-2ab796e217af None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:16.493985 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d56031-60a9-4172-9fc5-a53706eeaa90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:16.494785 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d56031-60a9-4172-9fc5-a53706eeaa90 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:16.495188 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d56031-60a9-4172-9fc5-a53706eeaa90 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:16.495575 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d56031-60a9-4172-9fc5-a53706eeaa90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:16.501227 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d962b-62fc-4a27-b16f-4fd9ee3a19f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:16.502225 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d962b-62fc-4a27-b16f-4fd9ee3a19f2 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:16.502687 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d962b-62fc-4a27-b16f-4fd9ee3a19f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:16.503675 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d962b-62fc-4a27-b16f-4fd9ee3a19f2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:16.504554 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27d962b-62fc-4a27-b16f-4fd9ee3a19f2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:16.506884 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27d962b-62fc-4a27-b16f-4fd9ee3a19f2 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:16.507010 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d56031-60a9-4172-9fc5-a53706eeaa90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:16.508533 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d56031-60a9-4172-9fc5-a53706eeaa90 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:16.518349 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-c27d962b-62fc-4a27-b16f-4fd9ee3a19f2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 13 12:58:16.518349 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:16.518349 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 13 12:58:16.518349 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 13 12:58:16.518349 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 13 12:58:16.518349 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 13 12:58:16.518349 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 13 12:58:16.518349 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:16.518349 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 13 12:58:16.518349 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:16.518349 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 12:58:16.518349 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:16.518349 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:16.518349 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:16.518349 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:16.518349 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:16.518349 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:16.519400 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:16.519400 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:16.519400 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:16.519400 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:16.519400 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:16.519400 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:16.519400 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 13 12:58:16.519400 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 13 12:58:16.519400 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 13 12:58:16.519400 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 13 12:58:16.519400 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:16.519400 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:16.519400 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:16.519400 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:16.519400 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:16.519400 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:16.520391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:16.520391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:16.520391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:16.520391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:16.520391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:16.520391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:16.520391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:16.520391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:16.520391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:16.520391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:16.520391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 13 12:58:16.520391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:16.520391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 13 12:58:16.520391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:16.520391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 347/698] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Oct 13 12:58:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:16.524383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-a7d56031-60a9-4172-9fc5-a53706eeaa90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:58:16.524383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:16.524383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:16.524383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:16.524383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:16.524383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:16.524383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:16.524383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:16.524383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:16.524383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:16.524383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:16.524383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:16.524383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:58:16.524383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:58:16.524383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:58:16.524383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:58:16.525879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:16.525879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:16.525879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:16.525879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:16.525879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:16.525879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:16.525879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:16.525879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:16.525879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:16.525879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:16.525879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:16.525879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:16.525879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:16.525879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:16.525879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:16.525879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:16.525879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:58:16.526788 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:58:16.526788 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:58:16.526788 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:16.528644 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d962b-62fc-4a27-b16f-4fd9ee3a19f2 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:16.530564 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 352/699] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:58:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:16.531484 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d56031-60a9-4172-9fc5-a53706eeaa90 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:16.591687 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:16.643529 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:16.646524 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d962b-62fc-4a27-b16f-4fd9ee3a19f2 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:16.651591 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d56031-60a9-4172-9fc5-a53706eeaa90 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:16.687201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32015ff-07fb-47c9-8268-3fb581575e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:16.688102 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32015ff-07fb-47c9-8268-3fb581575e8f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:16.688614 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32015ff-07fb-47c9-8268-3fb581575e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:16.689188 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32015ff-07fb-47c9-8268-3fb581575e8f None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:16.689991 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32015ff-07fb-47c9-8268-3fb581575e8f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:16.690471 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32015ff-07fb-47c9-8268-3fb581575e8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:16.692243 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32015ff-07fb-47c9-8268-3fb581575e8f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:16.694252 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2665dc4-7011-48a5-b9df-5520c37207e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:16.694929 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2665dc4-7011-48a5-b9df-5520c37207e2 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:16.695284 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2665dc4-7011-48a5-b9df-5520c37207e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:16.695622 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2665dc4-7011-48a5-b9df-5520c37207e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:16.696321 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2665dc4-7011-48a5-b9df-5520c37207e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:16.696706 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2665dc4-7011-48a5-b9df-5520c37207e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:16.698699 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2665dc4-7011-48a5-b9df-5520c37207e2 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:16.700029 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 348/700] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Oct 13 12:58:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:16.706296 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32015ff-07fb-47c9-8268-3fb581575e8f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:16.717475 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 353/701] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:58:16.722509 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2665dc4-7011-48a5-b9df-5520c37207e2 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:16.820821 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32015ff-07fb-47c9-8268-3fb581575e8f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:16.838888 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:16.857391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdedc3-78da-45ae-8aa4-466a55a7d9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:16.858068 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdedc3-78da-45ae-8aa4-466a55a7d9ab None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:16.858444 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdedc3-78da-45ae-8aa4-466a55a7d9ab None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:16.859000 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdedc3-78da-45ae-8aa4-466a55a7d9ab None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:16.859548 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcdedc3-78da-45ae-8aa4-466a55a7d9ab None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:16.859944 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcdedc3-78da-45ae-8aa4-466a55a7d9ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:16.861441 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcdedc3-78da-45ae-8aa4-466a55a7d9ab None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:16.862296 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2665dc4-7011-48a5-b9df-5520c37207e2 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:16.866972 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 349/702] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Oct 13 12:58:16 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:58:16.876374 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdedc3-78da-45ae-8aa4-466a55a7d9ab None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:16.908931 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2eeb3e-78c1-4070-a454-7444f225e8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:16.909821 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2eeb3e-78c1-4070-a454-7444f225e8bb None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:16.910317 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2eeb3e-78c1-4070-a454-7444f225e8bb None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:16.911242 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2eeb3e-78c1-4070-a454-7444f225e8bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:16.928122 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2eeb3e-78c1-4070-a454-7444f225e8bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:16.929979 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2eeb3e-78c1-4070-a454-7444f225e8bb None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:16.945985 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-6f2eeb3e-78c1-4070-a454-7444f225e8bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:58:16.945985 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:16.945985 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:16.945985 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:16.945985 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:16.945985 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:16.945985 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:16.945985 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:16.945985 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:16.945985 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:16.945985 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:16.945985 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:16.945985 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 13 12:58:16.945985 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 13 12:58:16.945985 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 13 12:58:16.946977 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 13 12:58:16.946977 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:16.946977 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:16.946977 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:16.946977 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:16.946977 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:16.946977 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:16.946977 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:16.946977 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:16.946977 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:16.946977 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:16.946977 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:16.946977 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:16.946977 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:16.946977 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:16.946977 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:16.946977 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:16.947965 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 13 12:58:16.947965 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 13 12:58:16.947965 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 13 12:58:16.947965 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 13 12:58:16.947965 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 13 12:58:16.947965 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 13 12:58:16.947965 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:58:16.947965 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:16.948375 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 354/703] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:58:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:58:16.956002 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2eeb3e-78c1-4070-a454-7444f225e8bb None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:16.983392 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:16.996612 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdedc3-78da-45ae-8aa4-466a55a7d9ab None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:17.032079 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba251c-b75d-49ce-ac15-a8181fab2aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:17.032683 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba251c-b75d-49ce-ac15-a8181fab2aa2 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:17.032954 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba251c-b75d-49ce-ac15-a8181fab2aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:17.033219 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba251c-b75d-49ce-ac15-a8181fab2aa2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:17.033910 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eba251c-b75d-49ce-ac15-a8181fab2aa2 None admin] RBAC: Adding query filter params (interface=public, service_id=a87502df6549448fb40f8d01db4e0e60, region_id=RegionOne) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:17.034211 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eba251c-b75d-49ce-ac15-a8181fab2aa2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:17.035592 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eba251c-b75d-49ce-ac15-a8181fab2aa2 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:17.045960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 350/704] 10.222.0.23 () {56 vars in 1251 bytes} [Thu Oct 13 12:58:16 2022] GET /identity/v3/endpoints?service_id=a87502df6549448fb40f8d01db4e0e60&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:17.052012 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba251c-b75d-49ce-ac15-a8181fab2aa2 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:17.084027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2eeb3e-78c1-4070-a454-7444f225e8bb None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:17.115949 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:17.129329 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c3451-644e-440d-9239-ed7a3c51d839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:17.130230 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c3451-644e-440d-9239-ed7a3c51d839 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:17.130752 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c3451-644e-440d-9239-ed7a3c51d839 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:17.131250 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c3451-644e-440d-9239-ed7a3c51d839 None admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:17.132169 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c3451-644e-440d-9239-ed7a3c51d839 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:17.133070 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c3451-644e-440d-9239-ed7a3c51d839 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:17.135091 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c3451-644e-440d-9239-ed7a3c51d839 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:17.151922 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 355/705] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 13 12:58:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:17.163166 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c3451-644e-440d-9239-ed7a3c51d839 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:17.163344 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba251c-b75d-49ce-ac15-a8181fab2aa2 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:17.202966 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9680e-b001-496f-b6e1-85d81b5953a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:17.203627 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9680e-b001-496f-b6e1-85d81b5953a0 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:17.203903 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9680e-b001-496f-b6e1-85d81b5953a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:17.204171 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9680e-b001-496f-b6e1-85d81b5953a0 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:17.204914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9680e-b001-496f-b6e1-85d81b5953a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:17.206325 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9680e-b001-496f-b6e1-85d81b5953a0 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:17.219153 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 351/706] 10.222.0.23 () {56 vars in 1087 bytes} [Thu Oct 13 12:58:17 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 12:58:17.274871 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:17.304648 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f1345-446a-4d08-b236-0560acf5628a None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:17.304972 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f1345-446a-4d08-b236-0560acf5628a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:17.305208 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f1345-446a-4d08-b236-0560acf5628a None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:17.306942 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 352/707] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 13 12:58:17.310308 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c3451-644e-440d-9239-ed7a3c51d839 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:17.312879 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636193b9-3416-449e-826c-ad4f773f41ab None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:17.313162 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636193b9-3416-449e-826c-ad4f773f41ab None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:17.313432 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636193b9-3416-449e-826c-ad4f773f41ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:17.333589 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-636193b9-3416-449e-826c-ad4f773f41ab None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:17.335223 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-636193b9-3416-449e-826c-ad4f773f41ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:17.350057 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976facba-ccda-4928-be18-c3ab4074c297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:17.350814 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976facba-ccda-4928-be18-c3ab4074c297 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:17.351189 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976facba-ccda-4928-be18-c3ab4074c297 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:17.351530 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976facba-ccda-4928-be18-c3ab4074c297 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:17.352355 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976facba-ccda-4928-be18-c3ab4074c297 None admin] RBAC: Adding query filter params (role.id=13498f16883f4cf0b7d010ec7e4d0866, scope.project.id=02308892635844b088ecdcb5413a21a0, user.id=b6ae674f0efc445fa970fca64e13893a) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:17.352726 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976facba-ccda-4928-be18-c3ab4074c297 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:17.354887 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976facba-ccda-4928-be18-c3ab4074c297 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:17.361611 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 356/708] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 13 12:58:17 2022] GET /identity/v3/role_assignments?user.id=b6ae674f0efc445fa970fca64e13893a&scope.project.id=02308892635844b088ecdcb5413a21a0&role.id=13498f16883f4cf0b7d010ec7e4d0866 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:17.418714 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 353/709] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 13 12:58:17.684169 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976facba-ccda-4928-be18-c3ab4074c297 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:17.789139 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976facba-ccda-4928-be18-c3ab4074c297 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:17.823524 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734c933-128f-4f93-8523-1fd2141d4123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:17.824153 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734c933-128f-4f93-8523-1fd2141d4123 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:17.824428 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734c933-128f-4f93-8523-1fd2141d4123 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:17.824694 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734c933-128f-4f93-8523-1fd2141d4123 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:17.825381 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734c933-128f-4f93-8523-1fd2141d4123 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:17.826766 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734c933-128f-4f93-8523-1fd2141d4123 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:17.835673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-b734c933-128f-4f93-8523-1fd2141d4123 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 13 12:58:17.835673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:17.835673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 13 12:58:17.835673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 13 12:58:17.835673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 13 12:58:17.835673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 13 12:58:17.835673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 13 12:58:17.835673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:17.835673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 13 12:58:17.835673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:17.835673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 12:58:17.835673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:17.835673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:17.835673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:17.835673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:17.835673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:17.835673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:17.836690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:17.836690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:17.836690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:17.836690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:17.836690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:17.836690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:17.836690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 13 12:58:17.836690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 13 12:58:17.836690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 13 12:58:17.836690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 13 12:58:17.836690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:17.836690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:17.836690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:17.836690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:17.836690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:17.836690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:17.838085 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:17.838085 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:17.838085 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:17.838085 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:17.838085 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:17.838085 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:17.838085 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:17.838085 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:17.838085 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:17.838085 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:17.838085 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 13 12:58:17.838085 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:17.838085 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 13 12:58:17.838085 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:17.838085 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 357/710] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Oct 13 12:58:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:17.843469 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636193b9-3416-449e-826c-ad4f773f41ab None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:17.887729 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:17.957310 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636193b9-3416-449e-826c-ad4f773f41ab None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:17.991158 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca4190-5843-4b9b-b1f3-2ef1a7a1123b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:17.992140 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca4190-5843-4b9b-b1f3-2ef1a7a1123b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:17.992612 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca4190-5843-4b9b-b1f3-2ef1a7a1123b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:17.993070 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca4190-5843-4b9b-b1f3-2ef1a7a1123b None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:17.994424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ca4190-5843-4b9b-b1f3-2ef1a7a1123b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:17.994916 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ca4190-5843-4b9b-b1f3-2ef1a7a1123b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:17.996486 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ca4190-5843-4b9b-b1f3-2ef1a7a1123b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:18.001781 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 354/711] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Oct 13 12:58:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:18.010689 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734c933-128f-4f93-8523-1fd2141d4123 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:18.012088 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:18.114966 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734c933-128f-4f93-8523-1fd2141d4123 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:18.151146 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b460f8-931a-4fe8-a4c8-ed27829f40ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:18.152193 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b460f8-931a-4fe8-a4c8-ed27829f40ab None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:18.152666 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b460f8-931a-4fe8-a4c8-ed27829f40ab None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:18.153126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b460f8-931a-4fe8-a4c8-ed27829f40ab None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:18.153941 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b460f8-931a-4fe8-a4c8-ed27829f40ab None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:18.154430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b460f8-931a-4fe8-a4c8-ed27829f40ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:18.156088 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b460f8-931a-4fe8-a4c8-ed27829f40ab None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:18.161302 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 358/712] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Oct 13 12:58:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:18.168307 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca4190-5843-4b9b-b1f3-2ef1a7a1123b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:18.277596 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca4190-5843-4b9b-b1f3-2ef1a7a1123b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:18.279939 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:18.315675 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8c8bf-2f43-4ff4-beb6-3d5b021290e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:18.316350 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8c8bf-2f43-4ff4-beb6-3d5b021290e4 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:18.316628 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8c8bf-2f43-4ff4-beb6-3d5b021290e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:18.316898 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8c8bf-2f43-4ff4-beb6-3d5b021290e4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:18.317627 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a8c8bf-2f43-4ff4-beb6-3d5b021290e4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:18.319015 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a8c8bf-2f43-4ff4-beb6-3d5b021290e4 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:18.344572 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 355/713] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 12:58:18 2022] POST /identity/v3/endpoints => generated 349 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:58:18.718442 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6cbe0-24ed-405b-94f4-a061e3b2e836 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:18.718959 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6cbe0-24ed-405b-94f4-a061e3b2e836 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:18.719034 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6cbe0-24ed-405b-94f4-a061e3b2e836 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:18.720694 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 359/714] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:18.725846 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ea716-039c-4d94-8a4f-37dd9c4a0a78 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:18.726137 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ea716-039c-4d94-8a4f-37dd9c4a0a78 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:18.726419 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ea716-039c-4d94-8a4f-37dd9c4a0a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:18.745845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-265ea716-039c-4d94-8a4f-37dd9c4a0a78 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:18.747164 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-265ea716-039c-4d94-8a4f-37dd9c4a0a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:18.822588 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 356/715] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:58:19.090888 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6cbe0-24ed-405b-94f4-a061e3b2e836 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:19.161355 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:19.197378 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6cbe0-24ed-405b-94f4-a061e3b2e836 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:19.232161 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea59998a-723e-402e-8f2c-ccc32f322014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:19.232796 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59998a-723e-402e-8f2c-ccc32f322014 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:19.233078 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59998a-723e-402e-8f2c-ccc32f322014 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:19.233351 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59998a-723e-402e-8f2c-ccc32f322014 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:19.233937 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea59998a-723e-402e-8f2c-ccc32f322014 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:19.235353 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea59998a-723e-402e-8f2c-ccc32f322014 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:19.245113 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-ea59998a-723e-402e-8f2c-ccc32f322014 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 13 12:58:19.245113 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:19.245113 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 13 12:58:19.245113 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 13 12:58:19.245113 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 13 12:58:19.245113 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 13 12:58:19.245113 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 13 12:58:19.245113 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:19.245113 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 13 12:58:19.245113 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:19.245113 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 12:58:19.245113 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:19.245113 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:19.245113 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:19.245113 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:19.245113 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:19.245113 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:19.246167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:19.246167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:19.246167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:19.246167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:19.246167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:19.246167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:19.246167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 13 12:58:19.246167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 13 12:58:19.246167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 13 12:58:19.246167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 13 12:58:19.246167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:19.246167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:19.246167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:19.246167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:19.246167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:19.246167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:19.247442 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:19.247442 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:19.247442 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:19.247442 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:19.247442 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:19.247442 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:19.247442 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:19.247442 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:19.247442 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:19.247442 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:19.247442 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 13 12:58:19.247442 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:19.247442 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 13 12:58:19.247442 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:19.247442 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 360/716] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:58:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:19.252843 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ea716-039c-4d94-8a4f-37dd9c4a0a78 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:19.339394 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:19.356662 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ea716-039c-4d94-8a4f-37dd9c4a0a78 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:19.395295 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abbf2c-cf31-45e2-b6c3-68df8424a411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:19.395915 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abbf2c-cf31-45e2-b6c3-68df8424a411 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:19.396190 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abbf2c-cf31-45e2-b6c3-68df8424a411 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:19.396463 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abbf2c-cf31-45e2-b6c3-68df8424a411 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:19.397098 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74abbf2c-cf31-45e2-b6c3-68df8424a411 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:19.397400 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74abbf2c-cf31-45e2-b6c3-68df8424a411 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:19.398772 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74abbf2c-cf31-45e2-b6c3-68df8424a411 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:19.404710 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 357/717] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Oct 13 12:58:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:19.410049 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea59998a-723e-402e-8f2c-ccc32f322014 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:19.487715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14340d8-ddf1-4cc6-b658-1dd90d3d557c None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:19.488045 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14340d8-ddf1-4cc6-b658-1dd90d3d557c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:19.488269 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14340d8-ddf1-4cc6-b658-1dd90d3d557c None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:19.489854 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 358/718] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 13 12:58:19.496104 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3485b52-52e4-4d25-92fa-33ba842f917a None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:19.496380 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3485b52-52e4-4d25-92fa-33ba842f917a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:19.496646 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3485b52-52e4-4d25-92fa-33ba842f917a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:19.514410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea59998a-723e-402e-8f2c-ccc32f322014 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:19.519639 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b3485b52-52e4-4d25-92fa-33ba842f917a None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:19.520851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b3485b52-52e4-4d25-92fa-33ba842f917a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:19.552822 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:19.559788 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59320a66-fa0a-4d92-9e06-9e33aeef3cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:19.560508 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59320a66-fa0a-4d92-9e06-9e33aeef3cee None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:19.561176 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59320a66-fa0a-4d92-9e06-9e33aeef3cee None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:19.561464 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59320a66-fa0a-4d92-9e06-9e33aeef3cee None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:19.562186 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59320a66-fa0a-4d92-9e06-9e33aeef3cee None admin] RBAC: Adding query filter params (type=compute) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:19.562563 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59320a66-fa0a-4d92-9e06-9e33aeef3cee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:19.563998 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59320a66-fa0a-4d92-9e06-9e33aeef3cee None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:19.574746 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 361/719] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Oct 13 12:58:19 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:19.617668 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 359/720] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 13 12:58:19.669860 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba503630-f989-48a0-97f8-d9eb8cc902a5 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:19.670155 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba503630-f989-48a0-97f8-d9eb8cc902a5 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:19.670420 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba503630-f989-48a0-97f8-d9eb8cc902a5 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:19.672218 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 362/721] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:19.681684 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df69761-74fd-4f75-908c-dcf2b585784d None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:19.681684 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df69761-74fd-4f75-908c-dcf2b585784d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:19.681684 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df69761-74fd-4f75-908c-dcf2b585784d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:19.700078 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8df69761-74fd-4f75-908c-dcf2b585784d None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:19.701537 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8df69761-74fd-4f75-908c-dcf2b585784d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:19.777353 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 360/722] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:58:19.893299 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba503630-f989-48a0-97f8-d9eb8cc902a5 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:20.001848 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba503630-f989-48a0-97f8-d9eb8cc902a5 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:20.036113 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7711d-958e-4f02-a9e5-68f526be30d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:20.036745 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7711d-958e-4f02-a9e5-68f526be30d5 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:20.037023 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7711d-958e-4f02-a9e5-68f526be30d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:20.037295 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7711d-958e-4f02-a9e5-68f526be30d5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:20.037873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce7711d-958e-4f02-a9e5-68f526be30d5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:20.039311 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce7711d-958e-4f02-a9e5-68f526be30d5 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:20.048761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-1ce7711d-958e-4f02-a9e5-68f526be30d5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 13 12:58:20.048761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:20.048761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 13 12:58:20.048761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 13 12:58:20.048761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 13 12:58:20.048761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 13 12:58:20.048761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 13 12:58:20.048761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:20.048761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 13 12:58:20.048761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:20.048761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 12:58:20.048761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:20.048761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:20.048761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:20.048761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:20.048761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:20.048761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:20.049714 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:20.049714 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:20.049714 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:20.049714 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:20.049714 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:20.049714 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:20.049714 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 13 12:58:20.049714 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 13 12:58:20.049714 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 13 12:58:20.049714 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 13 12:58:20.049714 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:20.049714 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:20.049714 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:20.049714 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:20.049714 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:20.049714 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:20.050941 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:20.050941 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:20.050941 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:20.050941 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:20.050941 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:20.050941 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:20.050941 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:20.050941 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:20.050941 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:20.050941 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:20.050941 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 13 12:58:20.050941 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:20.050941 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 13 12:58:20.050941 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:20.050941 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 363/723] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Oct 13 12:58:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:20.056749 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df69761-74fd-4f75-908c-dcf2b585784d None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:20.069315 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:20.085106 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7711d-958e-4f02-a9e5-68f526be30d5 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:20.147580 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:20.162750 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df69761-74fd-4f75-908c-dcf2b585784d None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:20.196260 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7711d-958e-4f02-a9e5-68f526be30d5 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:20.198523 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f404be3f-a683-481f-9b66-edd7ea064e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:20.199157 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404be3f-a683-481f-9b66-edd7ea064e68 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:20.199435 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404be3f-a683-481f-9b66-edd7ea064e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:20.199705 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404be3f-a683-481f-9b66-edd7ea064e68 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:20.200390 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f404be3f-a683-481f-9b66-edd7ea064e68 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:20.201409 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f404be3f-a683-481f-9b66-edd7ea064e68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:20.202091 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f404be3f-a683-481f-9b66-edd7ea064e68 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:20.207290 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 361/724] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Oct 13 12:58:20 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:20.220263 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f404be3f-a683-481f-9b66-edd7ea064e68 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:20.237045 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f674f-93b9-4d0c-b8ec-eb6b3ba6e5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:20.238264 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f674f-93b9-4d0c-b8ec-eb6b3ba6e5f9 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:20.238696 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f674f-93b9-4d0c-b8ec-eb6b3ba6e5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:20.239296 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f674f-93b9-4d0c-b8ec-eb6b3ba6e5f9 None admin] PATH_INFO: `/v3/services/image` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:20.240337 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f674f-93b9-4d0c-b8ec-eb6b3ba6e5f9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:20.241877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f674f-93b9-4d0c-b8ec-eb6b3ba6e5f9 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:20.255631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-9c9f674f-93b9-4d0c-b8ec-eb6b3ba6e5f9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 13 12:58:20.255631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:20.255631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 13 12:58:20.255631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 13 12:58:20.255631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 13 12:58:20.255631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 13 12:58:20.255631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 13 12:58:20.255631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:20.255631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 13 12:58:20.255631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:20.255631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 12:58:20.255631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:20.255631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:20.255631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:20.255631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:20.255631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:20.255631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:20.256877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:20.256877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:20.256877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:20.256877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:20.256877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:20.256877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:20.256877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 13 12:58:20.256877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 13 12:58:20.256877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 13 12:58:20.256877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 13 12:58:20.256877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:20.256877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:20.256877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:20.256877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:20.256877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:20.256877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:20.258006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:20.258006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:20.258006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:20.258006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:20.258006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:20.258006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:20.258006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:20.258006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:20.258006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:20.258006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:20.258006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 13 12:58:20.258006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:20.258006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 13 12:58:20.258006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:20.268258 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 364/725] 10.222.0.23 () {56 vars in 1105 bytes} [Thu Oct 13 12:58:20 2022] GET /identity/v3/services/image => generated 86 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:58:20.269167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f674f-93b9-4d0c-b8ec-eb6b3ba6e5f9 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:20.322820 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:20.361086 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f404be3f-a683-481f-9b66-edd7ea064e68 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:20.386063 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f674f-93b9-4d0c-b8ec-eb6b3ba6e5f9 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:20.399714 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f852a-ac88-47ea-a088-6739f7cc7094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:20.400402 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f852a-ac88-47ea-a088-6739f7cc7094 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:20.400775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f852a-ac88-47ea-a088-6739f7cc7094 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:20.401143 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f852a-ac88-47ea-a088-6739f7cc7094 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:20.401891 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750f852a-ac88-47ea-a088-6739f7cc7094 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:20.402250 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750f852a-ac88-47ea-a088-6739f7cc7094 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:20.403738 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750f852a-ac88-47ea-a088-6739f7cc7094 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:20.413606 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 362/726] 10.222.0.23 () {56 vars in 1114 bytes} [Thu Oct 13 12:58:20 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:20.414642 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f852a-ac88-47ea-a088-6739f7cc7094 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:20.417322 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:20.430660 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58007b6f-075c-4e44-80a4-7352ccef8454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:20.431944 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58007b6f-075c-4e44-80a4-7352ccef8454 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:20.432367 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58007b6f-075c-4e44-80a4-7352ccef8454 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:20.432731 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58007b6f-075c-4e44-80a4-7352ccef8454 None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:20.433462 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58007b6f-075c-4e44-80a4-7352ccef8454 None admin] RBAC: Adding query filter params (name=image) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:20.433890 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58007b6f-075c-4e44-80a4-7352ccef8454 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:20.435431 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58007b6f-075c-4e44-80a4-7352ccef8454 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:20.445641 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 365/727] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:58:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:20.447895 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58007b6f-075c-4e44-80a4-7352ccef8454 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:20.544383 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f852a-ac88-47ea-a088-6739f7cc7094 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:20.580865 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcc3ea9-8758-4a84-b162-64d6d5cdb6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:20.581468 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc3ea9-8758-4a84-b162-64d6d5cdb6dd None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:20.581747 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc3ea9-8758-4a84-b162-64d6d5cdb6dd None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:20.582018 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc3ea9-8758-4a84-b162-64d6d5cdb6dd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:20.582692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcc3ea9-8758-4a84-b162-64d6d5cdb6dd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:20.584143 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcc3ea9-8758-4a84-b162-64d6d5cdb6dd None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:20.586127 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58007b6f-075c-4e44-80a4-7352ccef8454 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:20.593477 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:20.615508 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 363/728] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 12:58:20 2022] POST /identity/v3/endpoints => generated 346 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:58:20.629420 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3335387-1e5e-4840-8f16-69f3d39f282b None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:20.629707 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3335387-1e5e-4840-8f16-69f3d39f282b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:20.630219 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3335387-1e5e-4840-8f16-69f3d39f282b None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:20.631702 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 364/729] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 13 12:58:20.640630 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71804d78-e47f-4757-a538-85137b40acf9 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:20.640957 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71804d78-e47f-4757-a538-85137b40acf9 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:20.641232 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71804d78-e47f-4757-a538-85137b40acf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:20.648424 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9a8473-0ce0-4edd-8452-323d3353f0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:20.649052 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a8473-0ce0-4edd-8452-323d3353f0a9 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:20.649336 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a8473-0ce0-4edd-8452-323d3353f0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:20.649609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a8473-0ce0-4edd-8452-323d3353f0a9 None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:20.650348 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9a8473-0ce0-4edd-8452-323d3353f0a9 None admin] RBAC: Adding query filter params (type=image) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:20.650966 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9a8473-0ce0-4edd-8452-323d3353f0a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:20.653554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9a8473-0ce0-4edd-8452-323d3353f0a9 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:20.659482 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 366/730] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:58:20 2022] GET /identity/v3/services?type=image => generated 122 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:20.668171 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-71804d78-e47f-4757-a538-85137b40acf9 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:20.669423 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-71804d78-e47f-4757-a538-85137b40acf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:20.754558 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 365/731] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:58:21.035412 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9a8473-0ce0-4edd-8452-323d3353f0a9 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:21.087535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:21.141614 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9a8473-0ce0-4edd-8452-323d3353f0a9 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:21.176893 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6278d39b-64c7-4764-bc6d-c2c53e406caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:21.177840 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278d39b-64c7-4764-bc6d-c2c53e406caa None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:21.178390 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278d39b-64c7-4764-bc6d-c2c53e406caa None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:21.178853 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278d39b-64c7-4764-bc6d-c2c53e406caa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:21.179648 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6278d39b-64c7-4764-bc6d-c2c53e406caa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:21.181737 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6278d39b-64c7-4764-bc6d-c2c53e406caa None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:21.191609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-6278d39b-64c7-4764-bc6d-c2c53e406caa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 13 12:58:21.191609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:21.191609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 13 12:58:21.191609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 13 12:58:21.191609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 13 12:58:21.191609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 13 12:58:21.191609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 13 12:58:21.191609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:21.191609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 13 12:58:21.191609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:21.191609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 12:58:21.191609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:21.191609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:21.191609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:21.191609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:21.191609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:21.191609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:21.192646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:21.192646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:21.192646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:21.192646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:21.192646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:21.192646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:21.192646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 13 12:58:21.192646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 13 12:58:21.192646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 13 12:58:21.192646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 13 12:58:21.192646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:21.192646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:21.192646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:21.192646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:21.192646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:21.192646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:21.193499 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:21.193499 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:21.193499 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:21.193499 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:21.193499 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:21.193499 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:21.193499 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:21.193499 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:21.193499 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:21.193499 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:21.193499 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 13 12:58:21.193499 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:21.193499 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 13 12:58:21.193499 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:21.201426 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 367/732] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Oct 13 12:58:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:21.203010 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71804d78-e47f-4757-a538-85137b40acf9 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:21.305093 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71804d78-e47f-4757-a538-85137b40acf9 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:21.329386 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:21.340702 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334f231-996b-4a9e-bc95-d2b7e3fcd893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:21.341509 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334f231-996b-4a9e-bc95-d2b7e3fcd893 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:21.341797 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334f231-996b-4a9e-bc95-d2b7e3fcd893 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:21.342066 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334f231-996b-4a9e-bc95-d2b7e3fcd893 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:21.342767 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f334f231-996b-4a9e-bc95-d2b7e3fcd893 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:21.343101 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f334f231-996b-4a9e-bc95-d2b7e3fcd893 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:21.344476 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f334f231-996b-4a9e-bc95-d2b7e3fcd893 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:21.349833 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 366/733] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Oct 13 12:58:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:21.356072 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6278d39b-64c7-4764-bc6d-c2c53e406caa None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:21.462384 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6278d39b-64c7-4764-bc6d-c2c53e406caa None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:21.476903 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:21.501143 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349a8c3-185d-4f4c-af07-4ae5f0453cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:21.502085 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349a8c3-185d-4f4c-af07-4ae5f0453cb9 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:21.502579 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349a8c3-185d-4f4c-af07-4ae5f0453cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:21.503088 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349a8c3-185d-4f4c-af07-4ae5f0453cb9 None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:21.503906 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349a8c3-185d-4f4c-af07-4ae5f0453cb9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:21.504398 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349a8c3-185d-4f4c-af07-4ae5f0453cb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:21.505939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349a8c3-185d-4f4c-af07-4ae5f0453cb9 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:21.515152 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 368/734] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Oct 13 12:58:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:21.517782 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334f231-996b-4a9e-bc95-d2b7e3fcd893 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:21.628838 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334f231-996b-4a9e-bc95-d2b7e3fcd893 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:21.662690 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fc860-cd97-40d4-acdc-577db2f24c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:21.663316 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fc860-cd97-40d4-acdc-577db2f24c1c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:21.663595 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fc860-cd97-40d4-acdc-577db2f24c1c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:21.663863 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fc860-cd97-40d4-acdc-577db2f24c1c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:21.664562 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8fc860-cd97-40d4-acdc-577db2f24c1c None admin] RBAC: Adding query filter params (interface=admin, service_id=1ef6ce68fefb4a0598606730f59beecd, region_id=RegionOne) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:21.664865 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8fc860-cd97-40d4-acdc-577db2f24c1c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:21.666753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8fc860-cd97-40d4-acdc-577db2f24c1c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:21.672089 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 367/735] 10.222.0.23 () {56 vars in 1249 bytes} [Thu Oct 13 12:58:21 2022] GET /identity/v3/endpoints?service_id=1ef6ce68fefb4a0598606730f59beecd&interface=admin®ion_id=RegionOne => generated 193 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:21.677898 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349a8c3-185d-4f4c-af07-4ae5f0453cb9 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:21.785101 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349a8c3-185d-4f4c-af07-4ae5f0453cb9 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:21.818679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0034f7-1a49-4f94-b57f-fda9e2bae10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:21.819308 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0034f7-1a49-4f94-b57f-fda9e2bae10e None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:21.819586 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0034f7-1a49-4f94-b57f-fda9e2bae10e None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:21.819856 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0034f7-1a49-4f94-b57f-fda9e2bae10e None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:21.820457 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0034f7-1a49-4f94-b57f-fda9e2bae10e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:21.821871 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0034f7-1a49-4f94-b57f-fda9e2bae10e None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:21.828140 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 369/736] 10.222.0.23 () {56 vars in 1087 bytes} [Thu Oct 13 12:58:21 2022] GET /identity/v3/services => generated 1497 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 12:58:21.831008 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6cda6-ca0f-4a0a-8ec4-3705101e5e56 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:21.831330 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6cda6-ca0f-4a0a-8ec4-3705101e5e56 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:21.831603 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6cda6-ca0f-4a0a-8ec4-3705101e5e56 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:21.833203 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 368/737] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 13 12:58:21.839328 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af840420-e8b1-42a8-9f13-057c313f5710 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:21.839627 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af840420-e8b1-42a8-9f13-057c313f5710 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:21.839869 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af840420-e8b1-42a8-9f13-057c313f5710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:21.859633 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-af840420-e8b1-42a8-9f13-057c313f5710 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:21.860855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-af840420-e8b1-42a8-9f13-057c313f5710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:21.898896 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:21.938576 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 370/738] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:58:22.209007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c6cda6-ca0f-4a0a-8ec4-3705101e5e56 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:22.263468 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:22.313707 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c6cda6-ca0f-4a0a-8ec4-3705101e5e56 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:22.348337 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f268b95-e492-4265-85d2-0cc70dc6844e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:22.348947 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f268b95-e492-4265-85d2-0cc70dc6844e None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:22.349206 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f268b95-e492-4265-85d2-0cc70dc6844e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:22.349481 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f268b95-e492-4265-85d2-0cc70dc6844e None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:22.350144 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f268b95-e492-4265-85d2-0cc70dc6844e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:22.351529 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f268b95-e492-4265-85d2-0cc70dc6844e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:22.367938 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 369/739] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Oct 13 12:58:22 2022] POST /identity/v3/services => generated 242 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:58:22.930800 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5210c46-9166-4cbd-9f03-a14c62b7cc2c None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:22.931916 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5210c46-9166-4cbd-9f03-a14c62b7cc2c None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:22.931990 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5210c46-9166-4cbd-9f03-a14c62b7cc2c None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:22.933562 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 371/740] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:22.939327 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e2711-64ee-47c1-a724-b4617e0dafb5 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:22.939625 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e2711-64ee-47c1-a724-b4617e0dafb5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:22.939892 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e2711-64ee-47c1-a724-b4617e0dafb5 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:22.940249 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbc36b-8b23-4a21-9b64-a8e1db675b21 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:22.940534 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbc36b-8b23-4a21-9b64-a8e1db675b21 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:22.940811 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbc36b-8b23-4a21-9b64-a8e1db675b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:22.941447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 370/741] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:22.946227 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240557bf-e8ba-4e80-af11-d9842c8bef59 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:22.946511 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240557bf-e8ba-4e80-af11-d9842c8bef59 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:22.946823 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240557bf-e8ba-4e80-af11-d9842c8bef59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:22.959636 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-69bbc36b-8b23-4a21-9b64-a8e1db675b21 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:22.960831 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-69bbc36b-8b23-4a21-9b64-a8e1db675b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:22.964814 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-240557bf-e8ba-4e80-af11-d9842c8bef59 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:22.965980 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-240557bf-e8ba-4e80-af11-d9842c8bef59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:23.045861 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 372/742] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:22 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:58:23.055083 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 371/743] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:22 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 13 12:58:23.315340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bbc36b-8b23-4a21-9b64-a8e1db675b21 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:23.320312 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240557bf-e8ba-4e80-af11-d9842c8bef59 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:23.365338 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:23.425949 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:23.434484 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bbc36b-8b23-4a21-9b64-a8e1db675b21 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:23.438526 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240557bf-e8ba-4e80-af11-d9842c8bef59 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:23.475728 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b705634-b9ac-42c1-b5a9-732e9ba2cdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:23.476667 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b705634-b9ac-42c1-b5a9-732e9ba2cdae None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:23.477090 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b705634-b9ac-42c1-b5a9-732e9ba2cdae None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:23.477601 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b705634-b9ac-42c1-b5a9-732e9ba2cdae None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:23.478718 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa84a4c-562d-454f-9352-63b38d1c9459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:23.479138 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b705634-b9ac-42c1-b5a9-732e9ba2cdae None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:23.479542 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa84a4c-562d-454f-9352-63b38d1c9459 None admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:23.479918 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa84a4c-562d-454f-9352-63b38d1c9459 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:23.480290 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa84a4c-562d-454f-9352-63b38d1c9459 None admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:23.481032 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa84a4c-562d-454f-9352-63b38d1c9459 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:23.481266 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b705634-b9ac-42c1-b5a9-732e9ba2cdae None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:23.482681 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa84a4c-562d-454f-9352-63b38d1c9459 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:23.485719 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: WARNING keystone.server.flask.common [None req-5fa84a4c-562d-454f-9352-63b38d1c9459 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 13 12:58:23.500785 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 372/744] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Oct 13 12:58:23 2022] POST /identity/v3/services => generated 242 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 12:58:23.512664 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.sql.core [None req-5fa84a4c-562d-454f-9352-63b38d1c9459 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 13 12:58:23.512664 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 13 12:58:23.512664 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [parameters: {'id': '8d4e656605234563848951c95291c066', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 13 12:58:23.512664 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81520) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 13 12:58:23.531203 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-5fa84a4c-562d-454f-9352-63b38d1c9459 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8d4e656605234563848951c95291c066.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8d4e656605234563848951c95291c066. Oct 13 12:58:23.531203 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:23.531203 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 13 12:58:23.531203 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 13 12:58:23.531203 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 13 12:58:23.531203 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 13 12:58:23.531203 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 13 12:58:23.531203 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application result = self._query(query) Oct 13 12:58:23.531203 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 13 12:58:23.531203 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application conn.query(q) Oct 13 12:58:23.531203 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 13 12:58:23.531203 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 13 12:58:23.531203 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 13 12:58:23.532256 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application result.read() Oct 13 12:58:23.532256 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 13 12:58:23.532256 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 13 12:58:23.532256 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 13 12:58:23.532256 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 13 12:58:23.532256 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 13 12:58:23.532256 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 13 12:58:23.532256 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 13 12:58:23.532256 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 13 12:58:23.532256 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 13 12:58:23.532256 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:23.532256 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 13 12:58:23.532256 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:23.532256 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:23.532256 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 13 12:58:23.532256 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 13 12:58:23.532256 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 13 12:58:23.532256 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 13 12:58:23.533467 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 13 12:58:23.533467 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application next(self.gen) Oct 13 12:58:23.533467 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 13 12:58:23.533467 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application yield resource Oct 13 12:58:23.533467 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 13 12:58:23.533467 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application next(self.gen) Oct 13 12:58:23.533467 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 13 12:58:23.533467 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application self.session.rollback() Oct 13 12:58:23.533467 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 13 12:58:23.533467 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application self.force_reraise() Oct 13 12:58:23.533467 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 13 12:58:23.533467 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise self.value Oct 13 12:58:23.533467 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 13 12:58:23.533467 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 13 12:58:23.533467 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 13 12:58:23.533467 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application session.commit() Oct 13 12:58:23.533467 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 13 12:58:23.534410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 13 12:58:23.534410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 13 12:58:23.534410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application self._prepare_impl() Oct 13 12:58:23.534410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 13 12:58:23.534410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application self.session.flush() Oct 13 12:58:23.534410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 13 12:58:23.534410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application self._flush(objects) Oct 13 12:58:23.534410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 13 12:58:23.534410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 13 12:58:23.534410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 13 12:58:23.534410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application compat.raise_( Oct 13 12:58:23.534410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 13 12:58:23.534410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception Oct 13 12:58:23.534410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 13 12:58:23.534410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application flush_context.execute() Oct 13 12:58:23.534410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 13 12:58:23.534410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rec.execute(self) Oct 13 12:58:23.535399 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 13 12:58:23.535399 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 13 12:58:23.535399 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 13 12:58:23.535399 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 13 12:58:23.535399 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 13 12:58:23.535399 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 13 12:58:23.535399 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 13 12:58:23.535399 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 13 12:58:23.535399 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 13 12:58:23.535399 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 13 12:58:23.535399 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 13 12:58:23.535399 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 13 12:58:23.535399 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 13 12:58:23.535399 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 13 12:58:23.535399 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 13 12:58:23.536416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 13 12:58:23.536416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 13 12:58:23.536416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception Oct 13 12:58:23.536416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 13 12:58:23.536416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 13 12:58:23.536416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 13 12:58:23.536416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 13 12:58:23.536416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 13 12:58:23.536416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application result = self._query(query) Oct 13 12:58:23.536416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 13 12:58:23.536416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application conn.query(q) Oct 13 12:58:23.536416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 13 12:58:23.536416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 13 12:58:23.536416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 13 12:58:23.536416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application result.read() Oct 13 12:58:23.536416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 13 12:58:23.536416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 13 12:58:23.537450 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 13 12:58:23.537450 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 13 12:58:23.537450 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 13 12:58:23.537450 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 13 12:58:23.537450 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 13 12:58:23.537450 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 13 12:58:23.537450 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 13 12:58:23.537450 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 13 12:58:23.537450 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [parameters: {'id': '8d4e656605234563848951c95291c066', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 13 12:58:23.537450 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 13 12:58:23.537450 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:23.537450 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 12:58:23.537450 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:23.537450 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:23.537450 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 13 12:58:23.538340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 13 12:58:23.538340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 13 12:58:23.538340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 13 12:58:23.538340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 13 12:58:23.538340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:23.538340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 12:58:23.538340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:23.538340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:23.538340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:23.538340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:23.538340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:23.538340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:23.538340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:23.538340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:23.538340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:23.538340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:23.538340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:23.539324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:23.539324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 13 12:58:23.539324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 13 12:58:23.539324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:23.539324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:23.539324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 13 12:58:23.539324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 13 12:58:23.539324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8d4e656605234563848951c95291c066. Oct 13 12:58:23.539324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:23.542342 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 373/745] 10.222.0.23 () {60 vars in 1140 bytes} [Thu Oct 13 12:58:23 2022] POST /identity/v3/projects => generated 262 bytes in 227 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:23.548927 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b705634-b9ac-42c1-b5a9-732e9ba2cdae None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:23.665686 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:23.676918 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b705634-b9ac-42c1-b5a9-732e9ba2cdae None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:23.714332 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d330f-c73d-4d22-8bfe-32a1e2e3a3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:23.714970 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d330f-c73d-4d22-8bfe-32a1e2e3a3fb None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:23.715302 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d330f-c73d-4d22-8bfe-32a1e2e3a3fb None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:23.715578 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d330f-c73d-4d22-8bfe-32a1e2e3a3fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:23.722618 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1d330f-c73d-4d22-8bfe-32a1e2e3a3fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:23.724101 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1d330f-c73d-4d22-8bfe-32a1e2e3a3fb None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:23.736523 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-da1d330f-c73d-4d22-8bfe-32a1e2e3a3fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:58:23.736523 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:23.736523 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:23.736523 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:23.736523 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:23.736523 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:23.736523 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:23.736523 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:23.736523 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:23.736523 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:23.736523 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:23.736523 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:23.736523 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 13 12:58:23.736523 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 13 12:58:23.736523 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 13 12:58:23.737440 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 13 12:58:23.737440 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:23.737440 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:23.737440 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:23.737440 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:23.737440 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:23.737440 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:23.737440 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:23.737440 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:23.737440 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:23.737440 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:23.737440 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:23.737440 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:23.737440 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:23.737440 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:23.737440 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:23.737440 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:23.738324 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 13 12:58:23.738324 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 13 12:58:23.738324 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 13 12:58:23.738324 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 13 12:58:23.738324 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 13 12:58:23.738324 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 13 12:58:23.738324 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:58:23.738324 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:23.738324 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 373/746] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:58:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:23.744442 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa84a4c-562d-454f-9352-63b38d1c9459 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:23.854187 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa84a4c-562d-454f-9352-63b38d1c9459 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:23.889314 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499ce2f3-2ebb-43a0-8286-1a071e5e4d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:23.890268 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ce2f3-2ebb-43a0-8286-1a071e5e4d6a None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:23.890746 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ce2f3-2ebb-43a0-8286-1a071e5e4d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:23.891221 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ce2f3-2ebb-43a0-8286-1a071e5e4d6a None admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:23.892056 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499ce2f3-2ebb-43a0-8286-1a071e5e4d6a None admin] RBAC: Adding query filter params (name=service) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:23.892533 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499ce2f3-2ebb-43a0-8286-1a071e5e4d6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:23.894348 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499ce2f3-2ebb-43a0-8286-1a071e5e4d6a None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:23.916249 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 374/747] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Oct 13 12:58:23 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:24.115475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a987295-9993-4c8d-8abc-45af40270ae0 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:24.116145 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a987295-9993-4c8d-8abc-45af40270ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:24.116642 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a987295-9993-4c8d-8abc-45af40270ae0 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:24.118471 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 374/748] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:24.124245 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5c7e2-dff9-48c7-b28b-4a229b093483 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:24.125089 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5c7e2-dff9-48c7-b28b-4a229b093483 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:24.125557 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5c7e2-dff9-48c7-b28b-4a229b093483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:24.144942 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-f0d5c7e2-dff9-48c7-b28b-4a229b093483 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:24.146776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-f0d5c7e2-dff9-48c7-b28b-4a229b093483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:24.156274 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:24.226372 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 375/749] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:24 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:58:24.496578 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a987295-9993-4c8d-8abc-45af40270ae0 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:24.561568 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5fb00-0a2c-4b35-8287-05df55139b59 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:24.561942 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5fb00-0a2c-4b35-8287-05df55139b59 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:24.562224 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5fb00-0a2c-4b35-8287-05df55139b59 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:24.563892 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 376/750] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:24.569896 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d23da-7a4d-4edd-8861-0885e5512fe8 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:24.570187 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d23da-7a4d-4edd-8861-0885e5512fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:24.570462 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d23da-7a4d-4edd-8861-0885e5512fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:24.593476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-662d23da-7a4d-4edd-8861-0885e5512fe8 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:24.595118 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-662d23da-7a4d-4edd-8861-0885e5512fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:24.670865 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 377/751] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:24 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 13 12:58:24.723158 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a987295-9993-4c8d-8abc-45af40270ae0 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:24.759479 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab99e315-9897-41b3-8444-f7c0d653ba58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:24.760029 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99e315-9897-41b3-8444-f7c0d653ba58 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:24.760308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99e315-9897-41b3-8444-f7c0d653ba58 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:24.760583 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99e315-9897-41b3-8444-f7c0d653ba58 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:24.761168 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab99e315-9897-41b3-8444-f7c0d653ba58 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:24.762526 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab99e315-9897-41b3-8444-f7c0d653ba58 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:24.771306 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-ab99e315-9897-41b3-8444-f7c0d653ba58 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 13 12:58:24.771306 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:24.771306 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 13 12:58:24.771306 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 13 12:58:24.771306 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 13 12:58:24.771306 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 13 12:58:24.771306 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 13 12:58:24.771306 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:24.771306 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 13 12:58:24.771306 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:24.771306 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 12:58:24.771306 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:24.771306 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:24.771306 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:24.771306 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:24.771306 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:24.771306 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:24.772309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:24.772309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:24.772309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:24.772309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:24.772309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:24.772309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:24.772309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 13 12:58:24.772309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 13 12:58:24.772309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 13 12:58:24.772309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 13 12:58:24.772309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:24.772309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:24.772309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:24.772309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:24.772309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:24.772309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:24.773155 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:24.773155 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:24.773155 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:24.773155 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:24.773155 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:24.773155 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:24.773155 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:24.773155 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:24.773155 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:24.773155 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:24.773155 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 13 12:58:24.773155 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:24.773155 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 13 12:58:24.773155 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:24.773155 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 375/752] 10.222.0.23 () {56 vars in 1126 bytes} [Thu Oct 13 12:58:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:24.779125 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d23da-7a4d-4edd-8861-0885e5512fe8 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:24.850090 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:24.886288 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d23da-7a4d-4edd-8861-0885e5512fe8 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:24.923433 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41245d3-caf0-4109-be51-8a9a75c459fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:24.923433 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41245d3-caf0-4109-be51-8a9a75c459fd None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:24.923433 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41245d3-caf0-4109-be51-8a9a75c459fd None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:24.923433 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41245d3-caf0-4109-be51-8a9a75c459fd None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:24.923851 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41245d3-caf0-4109-be51-8a9a75c459fd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:24.924121 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41245d3-caf0-4109-be51-8a9a75c459fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:24.925427 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41245d3-caf0-4109-be51-8a9a75c459fd None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:24.932433 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 378/753] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Oct 13 12:58:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:24.938450 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab99e315-9897-41b3-8444-f7c0d653ba58 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:24.943781 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41245d3-caf0-4109-be51-8a9a75c459fd None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:24.959046 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:25.049715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab99e315-9897-41b3-8444-f7c0d653ba58 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:25.076009 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41245d3-caf0-4109-be51-8a9a75c459fd None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:25.087429 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea7d2e-129c-4907-90a1-19416205508c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:25.088287 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea7d2e-129c-4907-90a1-19416205508c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:25.088742 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea7d2e-129c-4907-90a1-19416205508c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:25.089189 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea7d2e-129c-4907-90a1-19416205508c None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:25.089954 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ea7d2e-129c-4907-90a1-19416205508c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:25.090674 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ea7d2e-129c-4907-90a1-19416205508c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:25.092722 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ea7d2e-129c-4907-90a1-19416205508c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:25.098299 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 376/754] 10.222.0.23 () {56 vars in 1122 bytes} [Thu Oct 13 12:58:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:25.104895 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea7d2e-129c-4907-90a1-19416205508c None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:25.115921 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837747a-4725-4bcb-a9bc-d0442aa4eacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:25.116555 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837747a-4725-4bcb-a9bc-d0442aa4eacb None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:25.116832 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837747a-4725-4bcb-a9bc-d0442aa4eacb None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:25.117099 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837747a-4725-4bcb-a9bc-d0442aa4eacb None admin] PATH_INFO: `/v3/services/compute` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:25.117678 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837747a-4725-4bcb-a9bc-d0442aa4eacb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:25.119811 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837747a-4725-4bcb-a9bc-d0442aa4eacb None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:25.122223 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:25.129393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-8837747a-4725-4bcb-a9bc-d0442aa4eacb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 13 12:58:25.129393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:25.129393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 13 12:58:25.129393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 13 12:58:25.129393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 13 12:58:25.129393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 13 12:58:25.129393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 13 12:58:25.129393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:25.129393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 13 12:58:25.129393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:25.129393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 12:58:25.129393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:25.129393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:25.129393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:25.129393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:25.129393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:25.129393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:25.130818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:25.130818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:25.130818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:25.130818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:25.130818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:25.130818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:25.130818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 13 12:58:25.130818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 13 12:58:25.130818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 13 12:58:25.130818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 13 12:58:25.130818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:25.130818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:25.130818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:25.130818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:25.130818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:25.130818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:25.131684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:25.131684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:25.131684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:25.131684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:25.131684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:25.131684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:25.131684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:25.131684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:25.131684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:25.131684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:25.131684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 13 12:58:25.131684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:25.131684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 13 12:58:25.131684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:25.134669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 379/755] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:58:24 2022] GET /identity/v3/services/compute => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:25.140897 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837747a-4725-4bcb-a9bc-d0442aa4eacb None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:25.201100 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:25.219901 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea7d2e-129c-4907-90a1-19416205508c None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:25.255679 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f14851a-fb29-4fed-9704-3ce944815276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:25.256285 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14851a-fb29-4fed-9704-3ce944815276 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:25.256563 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14851a-fb29-4fed-9704-3ce944815276 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:25.256835 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14851a-fb29-4fed-9704-3ce944815276 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:25.257507 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f14851a-fb29-4fed-9704-3ce944815276 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:25.258021 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837747a-4725-4bcb-a9bc-d0442aa4eacb None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:25.258855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f14851a-fb29-4fed-9704-3ce944815276 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:25.277731 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 377/756] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 12:58:25 2022] POST /identity/v3/endpoints => generated 325 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:58:25.309670 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95131c-b34a-4de0-8379-7b250e1e9702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:25.310339 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95131c-b34a-4de0-8379-7b250e1e9702 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:25.310876 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95131c-b34a-4de0-8379-7b250e1e9702 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:25.311184 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95131c-b34a-4de0-8379-7b250e1e9702 None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:25.311823 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c95131c-b34a-4de0-8379-7b250e1e9702 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:25.312170 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c95131c-b34a-4de0-8379-7b250e1e9702 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:25.313585 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c95131c-b34a-4de0-8379-7b250e1e9702 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:25.325494 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 380/757] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Oct 13 12:58:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:58:25.328218 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f14851a-fb29-4fed-9704-3ce944815276 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:25.447809 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f14851a-fb29-4fed-9704-3ce944815276 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:25.483177 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8323b65-529e-4f51-b08a-2797921cd452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:25.483784 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8323b65-529e-4f51-b08a-2797921cd452 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:25.484058 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8323b65-529e-4f51-b08a-2797921cd452 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:25.484322 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8323b65-529e-4f51-b08a-2797921cd452 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:25.484947 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8323b65-529e-4f51-b08a-2797921cd452 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:25.485239 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8323b65-529e-4f51-b08a-2797921cd452 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:25.486583 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8323b65-529e-4f51-b08a-2797921cd452 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:25.492172 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 378/758] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Oct 13 12:58:25 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:25.498272 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95131c-b34a-4de0-8379-7b250e1e9702 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:25.561351 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:25.606068 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95131c-b34a-4de0-8379-7b250e1e9702 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:25.642017 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44d6408-f3f9-4f2e-8ace-bb6205ad6988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:25.642624 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d6408-f3f9-4f2e-8ace-bb6205ad6988 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:25.642899 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d6408-f3f9-4f2e-8ace-bb6205ad6988 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:25.643187 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d6408-f3f9-4f2e-8ace-bb6205ad6988 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:25.643884 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44d6408-f3f9-4f2e-8ace-bb6205ad6988 None admin] RBAC: Adding query filter params (interface=public, service_id=e0946e8c4f7a4b0fac89f7dfd18f3128, region_id=RegionOne) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:25.644188 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44d6408-f3f9-4f2e-8ace-bb6205ad6988 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:25.645546 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44d6408-f3f9-4f2e-8ace-bb6205ad6988 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:25.651018 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 381/759] 10.222.0.23 () {56 vars in 1251 bytes} [Thu Oct 13 12:58:25 2022] GET /identity/v3/endpoints?service_id=e0946e8c4f7a4b0fac89f7dfd18f3128&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:58:25.656408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8323b65-529e-4f51-b08a-2797921cd452 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:25.665884 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:25.768602 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8323b65-529e-4f51-b08a-2797921cd452 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:25.805558 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba421d-073e-4e5d-9bfb-b0a845971435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:25.806728 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba421d-073e-4e5d-9bfb-b0a845971435 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:25.807004 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba421d-073e-4e5d-9bfb-b0a845971435 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:25.807315 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba421d-073e-4e5d-9bfb-b0a845971435 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:25.807919 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba421d-073e-4e5d-9bfb-b0a845971435 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:25.809276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba421d-073e-4e5d-9bfb-b0a845971435 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:25.816737 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 379/760] 10.222.0.23 () {56 vars in 1087 bytes} [Thu Oct 13 12:58:25 2022] GET /identity/v3/services => generated 1957 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 12:58:25.875471 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3ca1e-9243-4606-a97a-f943b73857b3 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:25.875821 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3ca1e-9243-4606-a97a-f943b73857b3 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:25.875996 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3ca1e-9243-4606-a97a-f943b73857b3 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:25.877414 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 382/761] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:25.883391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfeb8d5-8e53-4ef4-8e1e-bbaaecebbf94 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:25.883696 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfeb8d5-8e53-4ef4-8e1e-bbaaecebbf94 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:25.883977 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfeb8d5-8e53-4ef4-8e1e-bbaaecebbf94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:25.903622 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-7cfeb8d5-8e53-4ef4-8e1e-bbaaecebbf94 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:25.904852 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7cfeb8d5-8e53-4ef4-8e1e-bbaaecebbf94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:25.976217 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:25.983579 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 380/762] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:25 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:58:26.256803 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3ca1e-9243-4606-a97a-f943b73857b3 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:26.262308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2a375-271b-4309-bcff-413b5c1f2af1 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:26.262308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2a375-271b-4309-bcff-413b5c1f2af1 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:26.262308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2a375-271b-4309-bcff-413b5c1f2af1 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:26.262308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 381/763] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 13 12:58:26.268405 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ed663-1ed0-4cc4-89a5-95dc63fcd227 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:26.268929 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ed663-1ed0-4cc4-89a5-95dc63fcd227 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:26.269409 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ed663-1ed0-4cc4-89a5-95dc63fcd227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:26.288073 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-086ed663-1ed0-4cc4-89a5-95dc63fcd227 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:26.289271 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-086ed663-1ed0-4cc4-89a5-95dc63fcd227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:26.375082 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 382/764] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:26 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 13 12:58:26.375276 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3ca1e-9243-4606-a97a-f943b73857b3 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:26.398501 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:26.415801 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8beaa8-c1ee-49ea-ad71-7ea8a5b0762a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:26.416745 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8beaa8-c1ee-49ea-ad71-7ea8a5b0762a None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:26.417349 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8beaa8-c1ee-49ea-ad71-7ea8a5b0762a None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:26.417815 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8beaa8-c1ee-49ea-ad71-7ea8a5b0762a None admin] PATH_INFO: `/v3/services/image` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:26.418638 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8beaa8-c1ee-49ea-ad71-7ea8a5b0762a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:26.420245 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8beaa8-c1ee-49ea-ad71-7ea8a5b0762a None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:26.429393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-0e8beaa8-c1ee-49ea-ad71-7ea8a5b0762a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 13 12:58:26.429393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:26.429393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 13 12:58:26.429393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 13 12:58:26.429393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 13 12:58:26.429393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 13 12:58:26.429393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 13 12:58:26.429393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:26.429393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 13 12:58:26.429393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:26.429393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 12:58:26.429393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:26.429393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:26.429393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:26.429393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:26.429393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:26.429393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:26.430352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:26.430352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:26.430352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:26.430352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:26.430352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:26.430352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:26.430352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 13 12:58:26.430352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 13 12:58:26.430352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 13 12:58:26.430352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 13 12:58:26.430352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:26.430352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:26.430352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:26.430352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:26.430352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:26.430352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:26.431789 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:26.431789 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:26.431789 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:26.431789 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:26.431789 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:26.431789 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:26.431789 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:26.431789 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:26.431789 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:26.431789 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:26.431789 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 13 12:58:26.431789 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:26.431789 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 13 12:58:26.431789 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:26.434457 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 383/765] 10.222.0.23 () {56 vars in 1105 bytes} [Thu Oct 13 12:58:26 2022] GET /identity/v3/services/image => generated 86 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:26.447103 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086ed663-1ed0-4cc4-89a5-95dc63fcd227 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:26.548685 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086ed663-1ed0-4cc4-89a5-95dc63fcd227 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:26.579482 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:26.586116 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52227bca-5bc0-4efc-aaa8-f8fb7a513223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:26.586920 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52227bca-5bc0-4efc-aaa8-f8fb7a513223 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:26.587584 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52227bca-5bc0-4efc-aaa8-f8fb7a513223 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:26.588010 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52227bca-5bc0-4efc-aaa8-f8fb7a513223 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:26.588802 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52227bca-5bc0-4efc-aaa8-f8fb7a513223 None admin] RBAC: Adding query filter params (name=image) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:26.589245 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52227bca-5bc0-4efc-aaa8-f8fb7a513223 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:26.590745 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52227bca-5bc0-4efc-aaa8-f8fb7a513223 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:26.596123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 383/766] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:58:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:26.604369 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8beaa8-c1ee-49ea-ad71-7ea8a5b0762a None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:26.664027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52227bca-5bc0-4efc-aaa8-f8fb7a513223 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:26.718829 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8beaa8-c1ee-49ea-ad71-7ea8a5b0762a None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:26.769387 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130424e-34ee-49f2-b3ba-a96a778278e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:26.770414 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130424e-34ee-49f2-b3ba-a96a778278e1 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:26.770752 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130424e-34ee-49f2-b3ba-a96a778278e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:26.771324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130424e-34ee-49f2-b3ba-a96a778278e1 None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:26.772109 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130424e-34ee-49f2-b3ba-a96a778278e1 None admin] RBAC: Adding query filter params (type=image) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:26.772583 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130424e-34ee-49f2-b3ba-a96a778278e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:26.774559 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130424e-34ee-49f2-b3ba-a96a778278e1 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:26.778848 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52227bca-5bc0-4efc-aaa8-f8fb7a513223 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:26.780732 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 384/767] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:58:26 2022] GET /identity/v3/services?type=image => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:26.785992 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130424e-34ee-49f2-b3ba-a96a778278e1 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:26.825759 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959be2ad-99a4-46d0-8905-947ee84a8500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:26.826439 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959be2ad-99a4-46d0-8905-947ee84a8500 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:26.826789 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959be2ad-99a4-46d0-8905-947ee84a8500 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:26.827198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959be2ad-99a4-46d0-8905-947ee84a8500 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:26.827916 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959be2ad-99a4-46d0-8905-947ee84a8500 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:26.830354 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959be2ad-99a4-46d0-8905-947ee84a8500 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:26.832961 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: WARNING keystone.server.flask.common [None req-959be2ad-99a4-46d0-8905-947ee84a8500 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 13 12:58:26.848408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.sql.core [None req-959be2ad-99a4-46d0-8905-947ee84a8500 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 13 12:58:26.848408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 13 12:58:26.848408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [parameters: {'user_id': '56cf2007059c4c5e84fe83fff7e0d92f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 13 12:58:26.848408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81521) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 13 12:58:26.849131 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:26.859615 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-959be2ad-99a4-46d0-8905-947ee84a8500 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 13 12:58:26.859615 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:26.859615 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 13 12:58:26.859615 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 13 12:58:26.859615 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 13 12:58:26.859615 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 13 12:58:26.859615 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 13 12:58:26.859615 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result = self._query(query) Oct 13 12:58:26.859615 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 13 12:58:26.859615 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application conn.query(q) Oct 13 12:58:26.859615 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 13 12:58:26.859615 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 13 12:58:26.859615 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 13 12:58:26.859615 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result.read() Oct 13 12:58:26.859615 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 13 12:58:26.860475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 13 12:58:26.860475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 13 12:58:26.860475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 13 12:58:26.860475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 13 12:58:26.860475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 13 12:58:26.860475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 13 12:58:26.860475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 13 12:58:26.860475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 13 12:58:26.860475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:26.860475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 13 12:58:26.860475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:26.860475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:26.860475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 13 12:58:26.860475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 13 12:58:26.860475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 13 12:58:26.860475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 13 12:58:26.860475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 13 12:58:26.860475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application next(self.gen) Oct 13 12:58:26.861391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 13 12:58:26.861391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application yield resource Oct 13 12:58:26.861391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 13 12:58:26.861391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application next(self.gen) Oct 13 12:58:26.861391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 13 12:58:26.861391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.session.rollback() Oct 13 12:58:26.861391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 13 12:58:26.861391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.force_reraise() Oct 13 12:58:26.861391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 13 12:58:26.861391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise self.value Oct 13 12:58:26.861391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 13 12:58:26.861391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 13 12:58:26.861391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 13 12:58:26.861391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application session.commit() Oct 13 12:58:26.861391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 13 12:58:26.861391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 13 12:58:26.861391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 13 12:58:26.863359 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._prepare_impl() Oct 13 12:58:26.863359 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 13 12:58:26.863359 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.session.flush() Oct 13 12:58:26.863359 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 13 12:58:26.863359 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._flush(objects) Oct 13 12:58:26.863359 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 13 12:58:26.863359 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 13 12:58:26.863359 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 13 12:58:26.863359 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application compat.raise_( Oct 13 12:58:26.863359 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 13 12:58:26.863359 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception Oct 13 12:58:26.863359 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 13 12:58:26.863359 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application flush_context.execute() Oct 13 12:58:26.863359 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 13 12:58:26.863359 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rec.execute(self) Oct 13 12:58:26.863359 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 13 12:58:26.863359 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 13 12:58:26.864741 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 13 12:58:26.864741 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 13 12:58:26.864741 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 13 12:58:26.864741 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 13 12:58:26.864741 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 13 12:58:26.864741 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 13 12:58:26.864741 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 13 12:58:26.864741 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 13 12:58:26.864741 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 13 12:58:26.864741 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 13 12:58:26.864741 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 13 12:58:26.864741 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 13 12:58:26.864741 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 13 12:58:26.864741 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 13 12:58:26.864741 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 13 12:58:26.864741 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception Oct 13 12:58:26.866119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 13 12:58:26.866119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 13 12:58:26.866119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 13 12:58:26.866119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 13 12:58:26.866119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 13 12:58:26.866119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result = self._query(query) Oct 13 12:58:26.866119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 13 12:58:26.866119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application conn.query(q) Oct 13 12:58:26.866119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 13 12:58:26.866119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 13 12:58:26.866119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 13 12:58:26.866119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result.read() Oct 13 12:58:26.866119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 13 12:58:26.866119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 13 12:58:26.866119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 13 12:58:26.866119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 13 12:58:26.871318 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 13 12:58:26.871318 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 13 12:58:26.871318 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 13 12:58:26.871318 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 13 12:58:26.871318 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 13 12:58:26.871318 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 13 12:58:26.871318 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [parameters: {'user_id': '56cf2007059c4c5e84fe83fff7e0d92f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 13 12:58:26.871318 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 13 12:58:26.871318 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:26.871318 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 12:58:26.871318 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:26.871318 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:26.871318 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:26.871318 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:26.871318 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:26.872185 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:26.872185 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:26.872185 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:26.872185 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:26.872185 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:26.872185 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:26.872185 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:26.872185 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 13 12:58:26.872185 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 13 12:58:26.872185 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:26.872185 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:26.872185 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 13 12:58:26.872185 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:58:26.872185 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 13 12:58:26.872185 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:58:26.872185 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 13 12:58:26.872185 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 13 12:58:26.873065 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 13 12:58:26.873065 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 13 12:58:26.873065 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 13 12:58:26.873065 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 13 12:58:26.873065 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 13 12:58:26.873065 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:26.873065 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 384/768] 10.222.0.23 () {60 vars in 1131 bytes} [Thu Oct 13 12:58:26 2022] POST /identity/v3/users => generated 160 bytes in 198 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:58:26.873065 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959be2ad-99a4-46d0-8905-947ee84a8500 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:26.916356 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130424e-34ee-49f2-b3ba-a96a778278e1 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:26.941815 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:26.963712 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84a3d8-7f47-45bb-bbcd-54022d70687a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:26.964540 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84a3d8-7f47-45bb-bbcd-54022d70687a None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:26.965256 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84a3d8-7f47-45bb-bbcd-54022d70687a None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:26.965529 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84a3d8-7f47-45bb-bbcd-54022d70687a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:26.966236 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84a3d8-7f47-45bb-bbcd-54022d70687a None admin] RBAC: Adding query filter params (interface=public, service_id=57251c3b8f594c73bcfb7593982052d2, region_id=RegionOne) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:26.966544 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84a3d8-7f47-45bb-bbcd-54022d70687a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:26.968310 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84a3d8-7f47-45bb-bbcd-54022d70687a None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:26.980563 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 385/769] 10.222.0.23 () {56 vars in 1251 bytes} [Thu Oct 13 12:58:26 2022] GET /identity/v3/endpoints?service_id=57251c3b8f594c73bcfb7593982052d2&interface=public®ion_id=RegionOne => generated 194 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:58:26.983593 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84a3d8-7f47-45bb-bbcd-54022d70687a None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:26.995492 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959be2ad-99a4-46d0-8905-947ee84a8500 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:27.031559 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcffb2e-ac5c-4d0f-be16-71390a85bb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:27.032576 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcffb2e-ac5c-4d0f-be16-71390a85bb7e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:27.033100 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcffb2e-ac5c-4d0f-be16-71390a85bb7e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:27.033572 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcffb2e-ac5c-4d0f-be16-71390a85bb7e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:27.041887 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcffb2e-ac5c-4d0f-be16-71390a85bb7e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:27.043861 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcffb2e-ac5c-4d0f-be16-71390a85bb7e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:27.055886 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-dbcffb2e-ac5c-4d0f-be16-71390a85bb7e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 13 12:58:27.055886 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:27.055886 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:27.055886 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:27.055886 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:27.055886 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:27.055886 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:27.055886 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:27.055886 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:27.055886 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:27.055886 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:27.055886 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:27.055886 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 13 12:58:27.055886 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 13 12:58:27.055886 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 13 12:58:27.055886 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 13 12:58:27.056851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:27.056851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:27.056851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 13 12:58:27.056851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:58:27.056851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 13 12:58:27.056851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:58:27.056851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:27.056851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:27.056851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:27.056851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:27.056851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:27.056851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:27.056851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:27.056851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:27.056851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:27.056851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:27.056851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:27.056851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:27.057793 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:27.057793 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:27.057793 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 13 12:58:27.057793 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 13 12:58:27.057793 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 13 12:58:27.057793 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 13 12:58:27.057793 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 13 12:58:27.057793 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 13 12:58:27.057793 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 13 12:58:27.057793 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:27.059960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 385/770] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:58:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:27.068853 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcffb2e-ac5c-4d0f-be16-71390a85bb7e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:27.092008 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84a3d8-7f47-45bb-bbcd-54022d70687a None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:27.109155 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:27.129058 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013633c-7b11-4d30-a4c0-43509943436a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:27.129670 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013633c-7b11-4d30-a4c0-43509943436a None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:27.129935 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013633c-7b11-4d30-a4c0-43509943436a None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:27.130210 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013633c-7b11-4d30-a4c0-43509943436a None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:27.130817 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5013633c-7b11-4d30-a4c0-43509943436a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:27.132216 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5013633c-7b11-4d30-a4c0-43509943436a None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:27.147128 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 386/771] 10.222.0.23 () {56 vars in 1087 bytes} [Thu Oct 13 12:58:26 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 12:58:27.189755 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcffb2e-ac5c-4d0f-be16-71390a85bb7e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:27.207509 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:27.228569 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d407af5-9ceb-4950-8623-509faf1df1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:27.229231 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d407af5-9ceb-4950-8623-509faf1df1e8 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:27.229523 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d407af5-9ceb-4950-8623-509faf1df1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:27.229794 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d407af5-9ceb-4950-8623-509faf1df1e8 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:27.230501 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d407af5-9ceb-4950-8623-509faf1df1e8 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:27.230861 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d407af5-9ceb-4950-8623-509faf1df1e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:27.232390 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d407af5-9ceb-4950-8623-509faf1df1e8 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:27.275607 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 386/772] 10.222.0.23 () {56 vars in 1101 bytes} [Thu Oct 13 12:58:27 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:58:27.565498 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa530667-9867-4c59-92bb-840c51b34129 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:27.566007 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa530667-9867-4c59-92bb-840c51b34129 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:27.566111 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa530667-9867-4c59-92bb-840c51b34129 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:27.567654 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 387/773] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:27.573113 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55163e6e-ecbb-4ac3-84ba-4e8880529c06 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:27.573401 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55163e6e-ecbb-4ac3-84ba-4e8880529c06 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:27.573672 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55163e6e-ecbb-4ac3-84ba-4e8880529c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:27.592010 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-55163e6e-ecbb-4ac3-84ba-4e8880529c06 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:27.594191 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-55163e6e-ecbb-4ac3-84ba-4e8880529c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:27.659464 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 387/774] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:27 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:58:27.924649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa530667-9867-4c59-92bb-840c51b34129 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:28.032752 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa530667-9867-4c59-92bb-840c51b34129 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:28.048139 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c8f94-f81d-40c6-a9b2-898acf963eed None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:28.048677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c8f94-f81d-40c6-a9b2-898acf963eed None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:28.048677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c8f94-f81d-40c6-a9b2-898acf963eed None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:28.050137 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 388/775] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 13 12:58:28.055016 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9dc3b6-46fb-408f-a9fc-8eb539823b47 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:28.055339 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9dc3b6-46fb-408f-a9fc-8eb539823b47 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:28.055644 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9dc3b6-46fb-408f-a9fc-8eb539823b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:28.067609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b121bf-1aba-4a07-a6dc-c9d026efd7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:28.068267 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b121bf-1aba-4a07-a6dc-c9d026efd7db None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:28.068550 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b121bf-1aba-4a07-a6dc-c9d026efd7db None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:28.068820 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b121bf-1aba-4a07-a6dc-c9d026efd7db None admin] PATH_INFO: `/v3/domains/default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:28.069997 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b121bf-1aba-4a07-a6dc-c9d026efd7db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:28.072050 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b121bf-1aba-4a07-a6dc-c9d026efd7db None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:28.075112 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 388/776] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:58:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:28.079536 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-bd9dc3b6-46fb-408f-a9fc-8eb539823b47 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:28.079866 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b121bf-1aba-4a07-a6dc-c9d026efd7db None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:28.080747 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-bd9dc3b6-46fb-408f-a9fc-8eb539823b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:28.130679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:28.150281 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 389/777] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:28 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 13 12:58:28.186662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b121bf-1aba-4a07-a6dc-c9d026efd7db None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:28.220674 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e0863d-0ced-4a0a-ba9d-14e1b69af5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:28.221509 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0863d-0ced-4a0a-ba9d-14e1b69af5fd None admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:28.221965 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0863d-0ced-4a0a-ba9d-14e1b69af5fd None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:28.222410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0863d-0ced-4a0a-ba9d-14e1b69af5fd None admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:28.223249 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e0863d-0ced-4a0a-ba9d-14e1b69af5fd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:28.224838 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e0863d-0ced-4a0a-ba9d-14e1b69af5fd None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:28.252033 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 389/778] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Oct 13 12:58:28 2022] POST /identity/v3/projects => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 12:58:28.419602 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9dc3b6-46fb-408f-a9fc-8eb539823b47 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:28.454828 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:28.525009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9dc3b6-46fb-408f-a9fc-8eb539823b47 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:28.559925 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cef3b2-0098-4236-8357-10d7062b0f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:28.561282 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cef3b2-0098-4236-8357-10d7062b0f7c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:28.561811 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cef3b2-0098-4236-8357-10d7062b0f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:28.562392 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cef3b2-0098-4236-8357-10d7062b0f7c None admin] PATH_INFO: `/v3/services/compute` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:28.563397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cef3b2-0098-4236-8357-10d7062b0f7c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:28.565131 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cef3b2-0098-4236-8357-10d7062b0f7c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:28.574520 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-78cef3b2-0098-4236-8357-10d7062b0f7c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 13 12:58:28.574520 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:28.574520 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 13 12:58:28.574520 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 13 12:58:28.574520 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 13 12:58:28.574520 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 13 12:58:28.574520 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 13 12:58:28.574520 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:28.574520 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 13 12:58:28.574520 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:28.574520 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 12:58:28.574520 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:28.574520 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:28.574520 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:28.574520 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:28.574520 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:28.574520 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:28.575708 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:28.575708 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:28.575708 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:28.575708 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:28.575708 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:28.575708 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:28.575708 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 13 12:58:28.575708 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 13 12:58:28.575708 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 13 12:58:28.575708 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 13 12:58:28.575708 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:28.575708 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:28.575708 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:28.575708 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:28.575708 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:28.575708 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:28.576898 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:28.576898 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:28.576898 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:28.576898 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:28.576898 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:28.576898 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:28.576898 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:28.576898 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:28.576898 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:28.576898 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:28.576898 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 13 12:58:28.576898 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:28.576898 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 13 12:58:28.576898 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:28.576898 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 390/779] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:58:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:28.582395 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e0863d-0ced-4a0a-ba9d-14e1b69af5fd None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:28.679972 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:28.683933 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e0863d-0ced-4a0a-ba9d-14e1b69af5fd None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:28.721358 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7710f80-6d59-4e72-9b54-38fbc03447f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:28.721893 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7710f80-6d59-4e72-9b54-38fbc03447f8 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:28.722173 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7710f80-6d59-4e72-9b54-38fbc03447f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:28.722445 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7710f80-6d59-4e72-9b54-38fbc03447f8 None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:28.723102 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7710f80-6d59-4e72-9b54-38fbc03447f8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:28.723397 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7710f80-6d59-4e72-9b54-38fbc03447f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:28.724758 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7710f80-6d59-4e72-9b54-38fbc03447f8 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:28.729880 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 390/780] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Oct 13 12:58:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:28.735304 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cef3b2-0098-4236-8357-10d7062b0f7c None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:28.833299 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:28.838558 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cef3b2-0098-4236-8357-10d7062b0f7c None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:28.872782 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0e9c7-0cda-4a18-9566-ed33b434763c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:28.873389 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0e9c7-0cda-4a18-9566-ed33b434763c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:28.873666 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0e9c7-0cda-4a18-9566-ed33b434763c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:28.873934 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0e9c7-0cda-4a18-9566-ed33b434763c None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:28.874555 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e0e9c7-0cda-4a18-9566-ed33b434763c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:28.874849 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e0e9c7-0cda-4a18-9566-ed33b434763c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:28.876238 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e0e9c7-0cda-4a18-9566-ed33b434763c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:28.886317 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 391/781] 10.222.0.23 () {56 vars in 1112 bytes} [Thu Oct 13 12:58:28 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:58:28.887536 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7710f80-6d59-4e72-9b54-38fbc03447f8 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:28.995976 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7710f80-6d59-4e72-9b54-38fbc03447f8 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:29.029737 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c834f81-2c1f-46ac-965b-a54b05834590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:29.030344 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c834f81-2c1f-46ac-965b-a54b05834590 None admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:29.030624 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c834f81-2c1f-46ac-965b-a54b05834590 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:29.030893 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c834f81-2c1f-46ac-965b-a54b05834590 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:29.031593 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c834f81-2c1f-46ac-965b-a54b05834590 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:29.032951 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c834f81-2c1f-46ac-965b-a54b05834590 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:29.059088 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 391/782] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 12:58:28 2022] POST /identity/v3/endpoints => generated 334 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 12:58:29.348614 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ff4a4-5fc5-4163-acea-410498e73b52 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:29.349157 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ff4a4-5fc5-4163-acea-410498e73b52 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:29.349157 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ff4a4-5fc5-4163-acea-410498e73b52 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:29.350618 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 392/783] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:29.355653 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8a5f6-af27-49a2-aa78-1ec8657079de None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:29.355941 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8a5f6-af27-49a2-aa78-1ec8657079de None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:29.356334 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8a5f6-af27-49a2-aa78-1ec8657079de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:29.374930 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-29e8a5f6-af27-49a2-aa78-1ec8657079de None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:29.376142 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-29e8a5f6-af27-49a2-aa78-1ec8657079de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:29.390718 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:29.455099 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 392/784] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:29 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:58:29.720814 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ff4a4-5fc5-4163-acea-410498e73b52 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:29.822256 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ff4a4-5fc5-4163-acea-410498e73b52 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:29.856351 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f0484-80bb-461f-b009-be13d7784436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:29.856980 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f0484-80bb-461f-b009-be13d7784436 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:29.857255 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f0484-80bb-461f-b009-be13d7784436 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:29.857522 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2f0484-80bb-461f-b009-be13d7784436 None admin] PATH_INFO: `/v3/services/image` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:29.858105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2f0484-80bb-461f-b009-be13d7784436 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:29.859487 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2f0484-80bb-461f-b009-be13d7784436 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:29.868005 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-dd2f0484-80bb-461f-b009-be13d7784436 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 13 12:58:29.868005 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:29.868005 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 13 12:58:29.868005 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 13 12:58:29.868005 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 13 12:58:29.868005 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 13 12:58:29.868005 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 13 12:58:29.868005 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:29.868005 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 13 12:58:29.868005 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:29.868005 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 12:58:29.868005 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:29.868005 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:29.868005 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:29.868005 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:29.868005 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:29.868005 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:29.869013 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:29.869013 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:29.869013 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:29.869013 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:29.869013 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:29.869013 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:29.869013 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 13 12:58:29.869013 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 13 12:58:29.869013 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 13 12:58:29.869013 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 13 12:58:29.869013 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:29.869013 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:29.869013 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:29.869013 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:29.869013 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:29.869013 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:29.869859 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:29.869859 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:29.869859 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:29.869859 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:29.869859 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:29.869859 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:29.869859 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:29.869859 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:29.869859 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:29.869859 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:29.869859 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 13 12:58:29.869859 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 12:58:29.869859 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 13 12:58:29.869859 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:29.874133 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 393/785] 10.222.0.23 () {56 vars in 1105 bytes} [Thu Oct 13 12:58:29 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:29.875417 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e8a5f6-af27-49a2-aa78-1ec8657079de None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:29.985549 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e8a5f6-af27-49a2-aa78-1ec8657079de None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:30.020935 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867258f-b33a-4122-98bf-346668c8779a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:30.021551 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867258f-b33a-4122-98bf-346668c8779a None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:30.021830 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867258f-b33a-4122-98bf-346668c8779a None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:30.022150 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867258f-b33a-4122-98bf-346668c8779a None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:30.022788 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6867258f-b33a-4122-98bf-346668c8779a None admin] RBAC: Adding query filter params (name=image) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:30.023115 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6867258f-b33a-4122-98bf-346668c8779a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:30.025017 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6867258f-b33a-4122-98bf-346668c8779a None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:30.034450 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 393/786] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:58:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:30.035722 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f0484-80bb-461f-b009-be13d7784436 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:30.064407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:30.136965 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2f0484-80bb-461f-b009-be13d7784436 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:30.170287 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f9b7d5-dae6-4126-9ae0-8af360cb5cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:30.170887 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9b7d5-dae6-4126-9ae0-8af360cb5cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:30.171203 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9b7d5-dae6-4126-9ae0-8af360cb5cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:30.171482 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9b7d5-dae6-4126-9ae0-8af360cb5cb1 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:30.172105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f9b7d5-dae6-4126-9ae0-8af360cb5cb1 None admin] RBAC: Adding query filter params (type=image) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:30.172407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f9b7d5-dae6-4126-9ae0-8af360cb5cb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:30.173777 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f9b7d5-dae6-4126-9ae0-8af360cb5cb1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:30.178658 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 394/787] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:58:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:58:30.184201 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867258f-b33a-4122-98bf-346668c8779a None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:30.197503 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:30.284451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867258f-b33a-4122-98bf-346668c8779a None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:30.317502 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19277ba-b8fc-4df7-bfed-266215238257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:30.318115 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19277ba-b8fc-4df7-bfed-266215238257 None admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:30.318396 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19277ba-b8fc-4df7-bfed-266215238257 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:30.318672 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19277ba-b8fc-4df7-bfed-266215238257 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:30.319388 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19277ba-b8fc-4df7-bfed-266215238257 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:30.320800 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19277ba-b8fc-4df7-bfed-266215238257 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:30.344221 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 394/788] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 12:58:30 2022] POST /identity/v3/endpoints => generated 327 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:58:30.472510 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c1829-41a2-4ee7-9ea1-9c73b4d2f746 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:30.473068 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c1829-41a2-4ee7-9ea1-9c73b4d2f746 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:30.473068 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c1829-41a2-4ee7-9ea1-9c73b4d2f746 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:30.474606 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 395/789] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:30.480211 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6eb12-2127-453e-8bf6-36b16cc689b4 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:30.480506 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6eb12-2127-453e-8bf6-36b16cc689b4 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:30.480784 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f6eb12-2127-453e-8bf6-36b16cc689b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:30.500995 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-34f6eb12-2127-453e-8bf6-36b16cc689b4 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:30.502239 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-34f6eb12-2127-453e-8bf6-36b16cc689b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:30.566573 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:30.582318 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 395/790] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:58:30.851506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c1829-41a2-4ee7-9ea1-9c73b4d2f746 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:30.948884 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:30.959804 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c1829-41a2-4ee7-9ea1-9c73b4d2f746 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:30.993677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c4bd7f-ef19-4df0-9628-d2599c5509f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:30.994304 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4bd7f-ef19-4df0-9628-d2599c5509f5 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:30.994580 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4bd7f-ef19-4df0-9628-d2599c5509f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:30.994851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4bd7f-ef19-4df0-9628-d2599c5509f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:30.995981 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c4bd7f-ef19-4df0-9628-d2599c5509f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:30.997405 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c4bd7f-ef19-4df0-9628-d2599c5509f5 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:30.999145 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 396/791] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:58:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:31.004601 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f6eb12-2127-453e-8bf6-36b16cc689b4 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:31.102917 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f6eb12-2127-453e-8bf6-36b16cc689b4 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:31.136986 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7a855-ce1b-4108-aabf-52b48aafeab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:31.137598 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7a855-ce1b-4108-aabf-52b48aafeab7 None admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:31.137875 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7a855-ce1b-4108-aabf-52b48aafeab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:31.138144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7a855-ce1b-4108-aabf-52b48aafeab7 None admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:31.139099 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7a855-ce1b-4108-aabf-52b48aafeab7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:31.140532 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7a855-ce1b-4108-aabf-52b48aafeab7 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:31.164142 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 396/792] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 13 12:58:31 2022] POST /identity/v3/users => generated 285 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:58:31.291835 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7d652-4f54-4e55-98c9-9afd1ae0e42d None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:31.292175 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7d652-4f54-4e55-98c9-9afd1ae0e42d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:31.292383 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7d652-4f54-4e55-98c9-9afd1ae0e42d None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:31.293930 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 397/793] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:31.299603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0f320-3fa0-4bd9-8094-ffbb5b6582a2 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:31.299888 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0f320-3fa0-4bd9-8094-ffbb5b6582a2 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:31.300159 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0f320-3fa0-4bd9-8094-ffbb5b6582a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:31.319806 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-90c0f320-3fa0-4bd9-8094-ffbb5b6582a2 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:31.321017 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-90c0f320-3fa0-4bd9-8094-ffbb5b6582a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:31.367773 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:31.392911 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 397/794] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:58:31.658601 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7d652-4f54-4e55-98c9-9afd1ae0e42d None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:31.757249 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7d652-4f54-4e55-98c9-9afd1ae0e42d None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:31.790328 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d56fe3d-d37f-4bd3-8795-f8648bf252a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:31.790940 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56fe3d-d37f-4bd3-8795-f8648bf252a4 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:31.791251 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56fe3d-d37f-4bd3-8795-f8648bf252a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:31.791514 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56fe3d-d37f-4bd3-8795-f8648bf252a4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:31.801063 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d56fe3d-d37f-4bd3-8795-f8648bf252a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:58:31.805674 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-6d56fe3d-d37f-4bd3-8795-f8648bf252a4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 13 12:58:31.805674 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:31.805674 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:31.805674 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:31.805674 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:31.805674 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:31.805674 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:31.805674 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:31.805674 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:31.805674 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:31.805674 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:31.805674 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:31.805674 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:58:31.805674 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:58:31.805674 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:58:31.805674 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:58:31.806565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:58:31.806565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:58:31.806565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:58:31.806565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:58:31.806565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:31.806565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:31.806565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:31.806565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:31.806565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:31.806565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:31.806565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:31.806565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:31.806565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:31.806565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:31.806565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:31.806565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:31.807447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:31.807447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:31.807447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:31.807447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:31.807447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:58:31.807447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:58:31.807447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:58:31.807447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:58:31.807447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:58:31.807447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:58:31.807447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 13 12:58:31.807447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:31.811373 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 398/795] 10.222.0.23 () {56 vars in 1120 bytes} [Thu Oct 13 12:58:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:31.813015 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c0f320-3fa0-4bd9-8094-ffbb5b6582a2 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:31.914611 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c0f320-3fa0-4bd9-8094-ffbb5b6582a2 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:31.950813 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b6f13c-da53-4733-97f2-4b4a3522a9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:31.951693 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6f13c-da53-4733-97f2-4b4a3522a9f4 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:31.952111 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6f13c-da53-4733-97f2-4b4a3522a9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:31.952541 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6f13c-da53-4733-97f2-4b4a3522a9f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:31.953335 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b6f13c-da53-4733-97f2-4b4a3522a9f4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:31.953842 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b6f13c-da53-4733-97f2-4b4a3522a9f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:31.955349 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b6f13c-da53-4733-97f2-4b4a3522a9f4 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:31.964557 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 398/796] 10.222.0.23 () {56 vars in 1115 bytes} [Thu Oct 13 12:58:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:31.970317 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d56fe3d-d37f-4bd3-8795-f8648bf252a4 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:31.995123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:32.074917 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d56fe3d-d37f-4bd3-8795-f8648bf252a4 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:32.110241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61069fd-9314-4c31-b9ea-20cbfc414927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:32.110865 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61069fd-9314-4c31-b9ea-20cbfc414927 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:32.111170 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61069fd-9314-4c31-b9ea-20cbfc414927 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:32.111438 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61069fd-9314-4c31-b9ea-20cbfc414927 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:32.121670 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61069fd-9314-4c31-b9ea-20cbfc414927 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:32.123115 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61069fd-9314-4c31-b9ea-20cbfc414927 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:32.137515 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-d61069fd-9314-4c31-b9ea-20cbfc414927 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:58:32.137515 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:32.137515 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:32.137515 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:32.137515 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:32.137515 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:32.137515 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:32.137515 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:32.137515 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:32.137515 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:32.137515 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:32.137515 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:32.137515 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:58:32.137515 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:58:32.137515 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:58:32.137515 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:58:32.138663 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:32.138663 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:32.138663 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:32.138663 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:32.138663 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:32.138663 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:32.138663 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:32.138663 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:32.138663 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:32.138663 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:32.138663 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:32.138663 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:32.138663 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:32.138663 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:32.138663 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:32.138663 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:32.138663 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:58:32.139990 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:58:32.139990 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:58:32.139990 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:32.143513 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 399/797] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:58:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:32.144496 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b6f13c-da53-4733-97f2-4b4a3522a9f4 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:32.187072 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:32.245324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b6f13c-da53-4733-97f2-4b4a3522a9f4 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:32.278522 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f43c0-c5cf-49ca-b049-bf28fa17bc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:32.279187 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f43c0-c5cf-49ca-b049-bf28fa17bc37 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:32.279466 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f43c0-c5cf-49ca-b049-bf28fa17bc37 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:32.279780 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f43c0-c5cf-49ca-b049-bf28fa17bc37 None admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:32.280416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373f43c0-c5cf-49ca-b049-bf28fa17bc37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:32.280714 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373f43c0-c5cf-49ca-b049-bf28fa17bc37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:32.282065 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373f43c0-c5cf-49ca-b049-bf28fa17bc37 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:32.294244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 399/798] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:58:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:32.299850 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61069fd-9314-4c31-b9ea-20cbfc414927 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:32.369669 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:32.402738 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61069fd-9314-4c31-b9ea-20cbfc414927 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:32.444154 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a56dd-fc18-4c6f-8355-3655cfba8f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:32.444706 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a56dd-fc18-4c6f-8355-3655cfba8f2e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:32.444986 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a56dd-fc18-4c6f-8355-3655cfba8f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:32.445258 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a56dd-fc18-4c6f-8355-3655cfba8f2e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:32.453425 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a56dd-fc18-4c6f-8355-3655cfba8f2e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:32.454850 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a56dd-fc18-4c6f-8355-3655cfba8f2e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:32.465722 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-704a56dd-fc18-4c6f-8355-3655cfba8f2e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 13 12:58:32.465722 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:32.465722 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:32.465722 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:32.465722 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:32.465722 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:32.465722 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:32.465722 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:32.465722 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:32.465722 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:32.465722 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:32.465722 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:32.465722 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 13 12:58:32.465722 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 13 12:58:32.465722 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 13 12:58:32.465722 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 13 12:58:32.466652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:32.466652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:32.466652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 13 12:58:32.466652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:58:32.466652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 13 12:58:32.466652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:58:32.466652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:32.466652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:32.466652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:32.466652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:32.466652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:32.466652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:32.466652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:32.466652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:32.466652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:32.466652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:32.466652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:32.466652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:32.472430 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:32.472430 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:32.472430 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 13 12:58:32.472430 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 13 12:58:32.472430 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 13 12:58:32.472430 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 13 12:58:32.472430 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 13 12:58:32.472430 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 13 12:58:32.472430 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 13 12:58:32.472430 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:32.472430 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 400/799] 10.222.0.23 () {56 vars in 1093 bytes} [Thu Oct 13 12:58:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:32.474709 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f43c0-c5cf-49ca-b049-bf28fa17bc37 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:32.528811 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee02592-cf47-4b21-b70d-df0c6348ac3e None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:32.529093 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee02592-cf47-4b21-b70d-df0c6348ac3e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:32.529367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee02592-cf47-4b21-b70d-df0c6348ac3e None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:32.530963 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 401/800] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 13 12:58:32.535553 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de41f3-3c50-4f96-a772-60493ddd6fe8 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:32.535905 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de41f3-3c50-4f96-a772-60493ddd6fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:32.536186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de41f3-3c50-4f96-a772-60493ddd6fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:32.554228 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-42de41f3-3c50-4f96-a772-60493ddd6fe8 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:32.555549 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-42de41f3-3c50-4f96-a772-60493ddd6fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:32.579297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:32.582075 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f43c0-c5cf-49ca-b049-bf28fa17bc37 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:32.617683 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04394f13-1902-408a-8bdc-ff3e4f02ed81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:32.618314 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04394f13-1902-408a-8bdc-ff3e4f02ed81 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:32.618592 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04394f13-1902-408a-8bdc-ff3e4f02ed81 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:32.618862 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04394f13-1902-408a-8bdc-ff3e4f02ed81 None admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:32.619616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04394f13-1902-408a-8bdc-ff3e4f02ed81 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:32.619916 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04394f13-1902-408a-8bdc-ff3e4f02ed81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:32.621328 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04394f13-1902-408a-8bdc-ff3e4f02ed81 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:32.629330 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 402/801] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 13 12:58:32.640384 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 400/802] 10.222.0.23 () {56 vars in 1133 bytes} [Thu Oct 13 12:58:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:58:32.646310 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de41f3-3c50-4f96-a772-60493ddd6fe8 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:32.704849 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:32.745328 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de41f3-3c50-4f96-a772-60493ddd6fe8 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:32.779129 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7928b1c-ffd0-42ec-8565-2c40974d25a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:32.779856 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7928b1c-ffd0-42ec-8565-2c40974d25a7 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:32.780137 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7928b1c-ffd0-42ec-8565-2c40974d25a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:32.780400 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7928b1c-ffd0-42ec-8565-2c40974d25a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:32.790243 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7928b1c-ffd0-42ec-8565-2c40974d25a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:32.791722 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7928b1c-ffd0-42ec-8565-2c40974d25a7 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:32.805845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-f7928b1c-ffd0-42ec-8565-2c40974d25a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:58:32.805845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:32.805845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:32.805845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:32.805845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:32.805845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:32.805845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:32.805845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:32.805845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:32.805845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:32.805845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:32.805845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:32.805845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:58:32.805845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:58:32.805845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:58:32.805845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:58:32.806811 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:32.806811 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:32.806811 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:32.806811 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:32.806811 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:32.806811 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:32.806811 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:32.806811 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:32.806811 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:32.806811 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:32.806811 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:32.806811 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:32.806811 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:32.806811 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:32.806811 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:32.806811 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:32.806811 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:58:32.808031 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:58:32.808031 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:58:32.808031 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:32.811369 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 403/803] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:58:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:32.812943 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04394f13-1902-408a-8bdc-ff3e4f02ed81 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:32.900319 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7928b1c-ffd0-42ec-8565-2c40974d25a7 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:32.922069 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04394f13-1902-408a-8bdc-ff3e4f02ed81 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:32.961959 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd01c5b-f83b-4db9-b0c4-76b488318e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:32.962485 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd01c5b-f83b-4db9-b0c4-76b488318e41 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:32.962766 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd01c5b-f83b-4db9-b0c4-76b488318e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:32.963034 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd01c5b-f83b-4db9-b0c4-76b488318e41 None admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:32.963698 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd01c5b-f83b-4db9-b0c4-76b488318e41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:32.963997 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd01c5b-f83b-4db9-b0c4-76b488318e41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:32.965369 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd01c5b-f83b-4db9-b0c4-76b488318e41 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:32.976762 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 401/804] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:58:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:32.983178 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd01c5b-f83b-4db9-b0c4-76b488318e41 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:33.003753 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:33.011946 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7928b1c-ffd0-42ec-8565-2c40974d25a7 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:33.025885 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:33.048550 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6308f-b046-4c35-a928-9de9c388d5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:33.049158 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6308f-b046-4c35-a928-9de9c388d5f4 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:33.049437 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6308f-b046-4c35-a928-9de9c388d5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:33.049712 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6308f-b046-4c35-a928-9de9c388d5f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:33.050379 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d6308f-b046-4c35-a928-9de9c388d5f4 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:33.051761 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d6308f-b046-4c35-a928-9de9c388d5f4 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:33.060090 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.sql.core [None req-13d6308f-b046-4c35-a928-9de9c388d5f4 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 13 12:58:33.060090 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 13 12:58:33.060090 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [parameters: {'id': '8a783e8b9adb4471adcf2fff3a682fbe', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 13 12:58:33.060090 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81521) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 13 12:58:33.069325 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-13d6308f-b046-4c35-a928-9de9c388d5f4 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 13 12:58:33.069325 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:33.069325 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 13 12:58:33.069325 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 13 12:58:33.069325 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 13 12:58:33.069325 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 13 12:58:33.069325 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 13 12:58:33.069325 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result = self._query(query) Oct 13 12:58:33.069325 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 13 12:58:33.069325 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application conn.query(q) Oct 13 12:58:33.069325 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 13 12:58:33.069325 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 13 12:58:33.069325 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 13 12:58:33.069325 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result.read() Oct 13 12:58:33.070182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 13 12:58:33.070182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 13 12:58:33.070182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 13 12:58:33.070182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 13 12:58:33.070182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 13 12:58:33.070182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 13 12:58:33.070182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 13 12:58:33.070182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 13 12:58:33.070182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 13 12:58:33.070182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:33.070182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 13 12:58:33.070182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:33.070182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:33.070182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 13 12:58:33.070182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 13 12:58:33.070182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 13 12:58:33.070182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 13 12:58:33.070182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 13 12:58:33.072447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application next(self.gen) Oct 13 12:58:33.072447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 13 12:58:33.072447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application yield resource Oct 13 12:58:33.072447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 13 12:58:33.072447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application next(self.gen) Oct 13 12:58:33.072447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 13 12:58:33.072447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.session.rollback() Oct 13 12:58:33.072447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 13 12:58:33.072447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.force_reraise() Oct 13 12:58:33.072447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 13 12:58:33.072447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise self.value Oct 13 12:58:33.072447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 13 12:58:33.072447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 13 12:58:33.072447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 13 12:58:33.072447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application session.commit() Oct 13 12:58:33.072447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 13 12:58:33.072447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 13 12:58:33.073343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 13 12:58:33.073343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._prepare_impl() Oct 13 12:58:33.073343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 13 12:58:33.073343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.session.flush() Oct 13 12:58:33.073343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 13 12:58:33.073343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._flush(objects) Oct 13 12:58:33.073343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 13 12:58:33.073343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 13 12:58:33.073343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 13 12:58:33.073343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application compat.raise_( Oct 13 12:58:33.073343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 13 12:58:33.073343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception Oct 13 12:58:33.073343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 13 12:58:33.073343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application flush_context.execute() Oct 13 12:58:33.073343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 13 12:58:33.073343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rec.execute(self) Oct 13 12:58:33.073343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 13 12:58:33.074347 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 13 12:58:33.074347 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 13 12:58:33.074347 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 13 12:58:33.074347 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 13 12:58:33.074347 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 13 12:58:33.074347 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 13 12:58:33.074347 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 13 12:58:33.074347 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 13 12:58:33.074347 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 13 12:58:33.074347 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 13 12:58:33.074347 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 13 12:58:33.074347 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 13 12:58:33.074347 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 13 12:58:33.074347 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 13 12:58:33.074347 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 13 12:58:33.075178 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 13 12:58:33.075178 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception Oct 13 12:58:33.075178 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 13 12:58:33.075178 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 13 12:58:33.075178 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 13 12:58:33.075178 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 13 12:58:33.075178 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 13 12:58:33.075178 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result = self._query(query) Oct 13 12:58:33.075178 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 13 12:58:33.075178 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application conn.query(q) Oct 13 12:58:33.075178 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 13 12:58:33.075178 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 13 12:58:33.075178 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 13 12:58:33.075178 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result.read() Oct 13 12:58:33.075178 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 13 12:58:33.075178 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 13 12:58:33.075178 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 13 12:58:33.076068 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 13 12:58:33.076068 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 13 12:58:33.076068 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 13 12:58:33.076068 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 13 12:58:33.076068 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 13 12:58:33.076068 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 13 12:58:33.076068 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 13 12:58:33.076068 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [parameters: {'id': '8a783e8b9adb4471adcf2fff3a682fbe', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 13 12:58:33.076068 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 13 12:58:33.076068 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:33.076068 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 12:58:33.076068 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:33.076068 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:33.076068 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 13 12:58:33.076068 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 13 12:58:33.076874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 13 12:58:33.076874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 13 12:58:33.076874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 13 12:58:33.076874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:33.076874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 12:58:33.076874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:33.076874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:33.076874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:33.076874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:33.076874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:33.076874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:33.076874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:33.076874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:33.076874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:33.076874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:33.076874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:33.076874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:33.078182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 13 12:58:33.078182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 13 12:58:33.078182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:33.078182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:33.078182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 13 12:58:33.078182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 13 12:58:33.078182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:33.078182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:33.078182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 13 12:58:33.078182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 13 12:58:33.078182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 13 12:58:33.078182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:33.078182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 404/805] 10.222.0.23 () {60 vars in 1137 bytes} [Thu Oct 13 12:58:32 2022] POST /identity/v3/domains => generated 190 bytes in 171 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:58:33.078182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6308f-b046-4c35-a928-9de9c388d5f4 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:33.095916 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd01c5b-f83b-4db9-b0c4-76b488318e41 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:33.135471 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d2563-4e96-4af4-b1e6-3eb45436472f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:33.136332 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d2563-4e96-4af4-b1e6-3eb45436472f None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:33.136884 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d2563-4e96-4af4-b1e6-3eb45436472f None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:33.137314 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d2563-4e96-4af4-b1e6-3eb45436472f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:33.144947 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d2563-4e96-4af4-b1e6-3eb45436472f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:33.147314 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d2563-4e96-4af4-b1e6-3eb45436472f None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:33.159073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-784d2563-4e96-4af4-b1e6-3eb45436472f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:58:33.159073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:33.159073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:33.159073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:33.159073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:33.159073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:33.159073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:33.159073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:33.159073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:33.159073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:33.159073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:33.159073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:33.159073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 13 12:58:33.159073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 13 12:58:33.159073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 13 12:58:33.159961 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 13 12:58:33.159961 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:33.159961 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:33.159961 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:33.159961 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:33.159961 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:33.159961 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:33.159961 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:33.159961 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:33.159961 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:33.159961 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:33.159961 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:33.159961 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:33.159961 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:33.159961 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:33.159961 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:33.159961 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:33.161033 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 13 12:58:33.161033 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 13 12:58:33.161033 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 13 12:58:33.161033 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 13 12:58:33.161033 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 13 12:58:33.161033 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 13 12:58:33.161033 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:58:33.161033 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:33.161033 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 402/806] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:58:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:58:33.166002 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d2563-4e96-4af4-b1e6-3eb45436472f None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:33.182952 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6308f-b046-4c35-a928-9de9c388d5f4 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:33.220160 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b752c38d-d39e-4a50-988a-ad92d721d962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:33.220875 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752c38d-d39e-4a50-988a-ad92d721d962 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:33.221104 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752c38d-d39e-4a50-988a-ad92d721d962 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:33.221374 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752c38d-d39e-4a50-988a-ad92d721d962 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:33.235460 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b752c38d-d39e-4a50-988a-ad92d721d962 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:33.237254 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b752c38d-d39e-4a50-988a-ad92d721d962 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:33.249176 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:33.257575 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-b752c38d-d39e-4a50-988a-ad92d721d962 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:58:33.257575 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:33.257575 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:33.257575 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:33.257575 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:33.257575 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:33.257575 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:33.257575 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:33.257575 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:33.257575 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:33.257575 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:33.257575 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:33.257575 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:58:33.257575 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:58:33.257575 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:58:33.257575 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:58:33.258677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:33.258677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:33.258677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:33.258677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:33.258677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:33.258677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:33.258677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:33.258677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:33.258677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:33.258677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:33.258677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:33.258677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:33.258677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:33.258677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:33.258677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:33.258677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:33.258677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:58:33.260222 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:58:33.260222 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:58:33.260222 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:33.260222 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 405/807] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:58:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:58:33.265047 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b752c38d-d39e-4a50-988a-ad92d721d962 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:33.274683 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d2563-4e96-4af4-b1e6-3eb45436472f None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:33.294424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:33.313646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f31a6a-e172-4ae0-9518-466759d18aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:33.314373 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f31a6a-e172-4ae0-9518-466759d18aed None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:33.314657 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f31a6a-e172-4ae0-9518-466759d18aed None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:33.314931 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f31a6a-e172-4ae0-9518-466759d18aed None admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:33.315763 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f31a6a-e172-4ae0-9518-466759d18aed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:33.316072 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f31a6a-e172-4ae0-9518-466759d18aed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:33.317607 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f31a6a-e172-4ae0-9518-466759d18aed None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:33.330389 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 403/808] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 13 12:58:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:33.337103 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f31a6a-e172-4ae0-9518-466759d18aed None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:33.376739 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b752c38d-d39e-4a50-988a-ad92d721d962 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:33.409736 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd0e83-a788-41f0-abc1-b204d4c78a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:33.410323 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd0e83-a788-41f0-abc1-b204d4c78a42 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:33.410602 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd0e83-a788-41f0-abc1-b204d4c78a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:33.410879 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd0e83-a788-41f0-abc1-b204d4c78a42 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:33.411536 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdd0e83-a788-41f0-abc1-b204d4c78a42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:33.411835 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdd0e83-a788-41f0-abc1-b204d4c78a42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:33.413190 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdd0e83-a788-41f0-abc1-b204d4c78a42 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:33.424564 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 406/809] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:58:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:58:33.428696 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7fb41-87e9-4c6c-825e-27d5bde1c6b0 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:33.429142 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7fb41-87e9-4c6c-825e-27d5bde1c6b0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:33.429418 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7fb41-87e9-4c6c-825e-27d5bde1c6b0 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:33.432236 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 407/810] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:33.436898 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f31a6a-e172-4ae0-9518-466759d18aed None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:33.438286 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e40770a-41d5-459c-8d84-acc3bf02e209 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:33.438803 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e40770a-41d5-459c-8d84-acc3bf02e209 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:33.439105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e40770a-41d5-459c-8d84-acc3bf02e209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:33.461849 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-9e40770a-41d5-459c-8d84-acc3bf02e209 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:33.463099 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9e40770a-41d5-459c-8d84-acc3bf02e209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:33.477166 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df2cb32-ab13-48f5-8dea-db5510ede2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:33.478450 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2cb32-ab13-48f5-8dea-db5510ede2ea None admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:33.478800 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2cb32-ab13-48f5-8dea-db5510ede2ea None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:33.479181 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df2cb32-ab13-48f5-8dea-db5510ede2ea None admin] PATH_INFO: `/v3/projects/02308892635844b088ecdcb5413a21a0/users/12700d985a724595af630c81e63f86d7/roles/13498f16883f4cf0b7d010ec7e4d0866` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:33.481992 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df2cb32-ab13-48f5-8dea-db5510ede2ea None admin] RBAC: Authorizing `identity:create_grant(project_id=02308892635844b088ecdcb5413a21a0, user_id=12700d985a724595af630c81e63f86d7, role_id=13498f16883f4cf0b7d010ec7e4d0866)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:33.483600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df2cb32-ab13-48f5-8dea-db5510ede2ea None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:33.493054 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:33.501930 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 404/811] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 13 12:58:33 2022] PUT /identity/v3/projects/02308892635844b088ecdcb5413a21a0/users/12700d985a724595af630c81e63f86d7/roles/13498f16883f4cf0b7d010ec7e4d0866 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 12:58:33.545445 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 408/812] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 13 12:58:33.813952 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df2cb32-ab13-48f5-8dea-db5510ede2ea None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:34.025323 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df2cb32-ab13-48f5-8dea-db5510ede2ea None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:34.057631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70972d5e-0d79-407b-9951-6f32f6fdd46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:34.058250 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70972d5e-0d79-407b-9951-6f32f6fdd46d None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:34.058530 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70972d5e-0d79-407b-9951-6f32f6fdd46d None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:34.058799 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70972d5e-0d79-407b-9951-6f32f6fdd46d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:34.068854 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70972d5e-0d79-407b-9951-6f32f6fdd46d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:58:34.074315 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-70972d5e-0d79-407b-9951-6f32f6fdd46d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:58:34.074315 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:34.074315 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:34.074315 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:34.074315 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:34.074315 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:34.074315 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:34.074315 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:34.074315 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:34.074315 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:34.074315 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:34.074315 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:34.074315 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:58:34.074315 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:58:34.074315 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:58:34.074315 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:58:34.075419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:58:34.075419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:58:34.075419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:58:34.075419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:58:34.075419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:34.075419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:34.075419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:34.075419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:34.075419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:34.075419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:34.075419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:34.075419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:34.075419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:34.075419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:34.075419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:34.075419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:34.080261 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:34.080261 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:34.080261 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:34.080261 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:34.080261 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:58:34.080261 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:58:34.080261 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:58:34.080261 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:58:34.080261 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:58:34.080261 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:58:34.080261 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:58:34.080261 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:34.080261 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 405/813] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 13 12:58:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:34.081415 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e40770a-41d5-459c-8d84-acc3bf02e209 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:34.099270 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:34.175376 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e40770a-41d5-459c-8d84-acc3bf02e209 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:34.207699 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c27ef6d-01ce-4d12-8fc4-510141c129b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:34.208297 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27ef6d-01ce-4d12-8fc4-510141c129b2 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:34.208571 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27ef6d-01ce-4d12-8fc4-510141c129b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:34.208840 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27ef6d-01ce-4d12-8fc4-510141c129b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:34.209495 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c27ef6d-01ce-4d12-8fc4-510141c129b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:34.209790 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c27ef6d-01ce-4d12-8fc4-510141c129b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:34.211161 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c27ef6d-01ce-4d12-8fc4-510141c129b2 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:34.219230 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 409/814] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:58:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:34.224961 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70972d5e-0d79-407b-9951-6f32f6fdd46d None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:34.280684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:34.321430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70972d5e-0d79-407b-9951-6f32f6fdd46d None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:34.354101 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d128b74d-4625-4d87-b4c2-e80291bda270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:34.354719 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128b74d-4625-4d87-b4c2-e80291bda270 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:34.354992 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128b74d-4625-4d87-b4c2-e80291bda270 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:34.355283 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128b74d-4625-4d87-b4c2-e80291bda270 None admin] PATH_INFO: `/v3/users/780419c52cdb4fa09e1d24c263946307` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:34.368778 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d128b74d-4625-4d87-b4c2-e80291bda270 None admin] RBAC: Authorizing `identity:get_user(user_id=780419c52cdb4fa09e1d24c263946307)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:34.370268 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d128b74d-4625-4d87-b4c2-e80291bda270 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:34.372015 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 406/815] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:58:34 2022] GET /identity/v3/users/780419c52cdb4fa09e1d24c263946307 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:34.378050 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c27ef6d-01ce-4d12-8fc4-510141c129b2 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:34.455671 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:34.474237 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c27ef6d-01ce-4d12-8fc4-510141c129b2 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:34.506068 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908f54c-769e-467a-a3b4-3dc23e91a0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:34.506686 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908f54c-769e-467a-a3b4-3dc23e91a0fd None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:34.506963 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908f54c-769e-467a-a3b4-3dc23e91a0fd None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:34.507338 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908f54c-769e-467a-a3b4-3dc23e91a0fd None admin] PATH_INFO: `/v3/projects/21bac567038642bdb14df2ee5e66934c` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:34.508469 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6908f54c-769e-467a-a3b4-3dc23e91a0fd None admin] RBAC: Authorizing `identity:get_project(project_id=21bac567038642bdb14df2ee5e66934c)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:34.509897 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6908f54c-769e-467a-a3b4-3dc23e91a0fd None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:34.511541 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 410/816] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 13 12:58:34 2022] GET /identity/v3/projects/21bac567038642bdb14df2ee5e66934c => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:34.517400 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d128b74d-4625-4d87-b4c2-e80291bda270 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:34.613617 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d128b74d-4625-4d87-b4c2-e80291bda270 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:34.628985 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:34.645932 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74db79-f17a-462e-865e-9a9f485c4f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:34.646584 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74db79-f17a-462e-865e-9a9f485c4f3e None admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:34.646862 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74db79-f17a-462e-865e-9a9f485c4f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:34.647170 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74db79-f17a-462e-865e-9a9f485c4f3e None admin] PATH_INFO: `/v3/projects/21bac567038642bdb14df2ee5e66934c/users/780419c52cdb4fa09e1d24c263946307/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:34.649567 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc74db79-f17a-462e-865e-9a9f485c4f3e None admin] RBAC: Authorizing `identity:create_grant(project_id=21bac567038642bdb14df2ee5e66934c, user_id=780419c52cdb4fa09e1d24c263946307, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:34.650995 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc74db79-f17a-462e-865e-9a9f485c4f3e None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:34.664323 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 407/817] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 13 12:58:34 2022] PUT /identity/v3/projects/21bac567038642bdb14df2ee5e66934c/users/780419c52cdb4fa09e1d24c263946307/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 12:58:35.649455 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75404ecb-b3ab-4790-9d30-860c28a361bb None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:35.650009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75404ecb-b3ab-4790-9d30-860c28a361bb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:35.650009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75404ecb-b3ab-4790-9d30-860c28a361bb None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:35.651360 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 411/818] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:35.656549 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225c4f0-9135-4808-bb47-992b76d25afc None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:35.656940 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225c4f0-9135-4808-bb47-992b76d25afc None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:35.657209 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225c4f0-9135-4808-bb47-992b76d25afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:35.675519 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-5225c4f0-9135-4808-bb47-992b76d25afc None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:35.676685 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-5225c4f0-9135-4808-bb47-992b76d25afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:35.741313 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 408/819] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:58:36.008804 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75404ecb-b3ab-4790-9d30-860c28a361bb None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:36.104892 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75404ecb-b3ab-4790-9d30-860c28a361bb None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:36.136797 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d451a18b-fcf2-4eea-b005-fe619b1c1c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:36.137401 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451a18b-fcf2-4eea-b005-fe619b1c1c6b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:36.137750 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451a18b-fcf2-4eea-b005-fe619b1c1c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:36.138022 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451a18b-fcf2-4eea-b005-fe619b1c1c6b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:36.147450 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d451a18b-fcf2-4eea-b005-fe619b1c1c6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:58:36.152064 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-d451a18b-fcf2-4eea-b005-fe619b1c1c6b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 13 12:58:36.152064 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:36.152064 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:36.152064 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:36.152064 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:36.152064 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:36.152064 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:36.152064 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:36.152064 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:36.152064 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:36.152064 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:36.152064 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:36.152064 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:58:36.152064 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:58:36.152064 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:58:36.152064 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:58:36.152982 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:58:36.152982 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:58:36.152982 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:58:36.152982 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:58:36.152982 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:36.152982 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:36.152982 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:36.152982 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:36.152982 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:36.152982 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:36.152982 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:36.152982 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:36.152982 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:36.152982 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:36.152982 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:36.152982 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:36.153809 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:36.153809 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:36.153809 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:36.153809 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:36.153809 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:58:36.153809 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:58:36.153809 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:58:36.153809 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:58:36.153809 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:58:36.153809 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:58:36.153809 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 13 12:58:36.153809 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:36.153809 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 412/820] 10.222.0.23 () {56 vars in 1120 bytes} [Thu Oct 13 12:58:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:36.159077 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225c4f0-9135-4808-bb47-992b76d25afc None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:36.253259 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225c4f0-9135-4808-bb47-992b76d25afc None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:36.260297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:36.287468 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893a4ba-a02d-4124-9408-e487e44adf57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:36.288067 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893a4ba-a02d-4124-9408-e487e44adf57 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:36.288339 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893a4ba-a02d-4124-9408-e487e44adf57 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:36.288603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893a4ba-a02d-4124-9408-e487e44adf57 None admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:36.289258 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893a4ba-a02d-4124-9408-e487e44adf57 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:36.290077 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893a4ba-a02d-4124-9408-e487e44adf57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:36.291458 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893a4ba-a02d-4124-9408-e487e44adf57 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:36.299037 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 409/821] 10.222.0.23 () {56 vars in 1115 bytes} [Thu Oct 13 12:58:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:36.304914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d451a18b-fcf2-4eea-b005-fe619b1c1c6b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:36.316766 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:36.401450 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d451a18b-fcf2-4eea-b005-fe619b1c1c6b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:36.434959 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105957b5-d2cf-435e-9503-c4df6a89be65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:36.435853 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105957b5-d2cf-435e-9503-c4df6a89be65 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:36.436125 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105957b5-d2cf-435e-9503-c4df6a89be65 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:36.436392 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105957b5-d2cf-435e-9503-c4df6a89be65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:36.445870 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105957b5-d2cf-435e-9503-c4df6a89be65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:36.447315 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105957b5-d2cf-435e-9503-c4df6a89be65 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:36.460978 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-105957b5-d2cf-435e-9503-c4df6a89be65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:58:36.460978 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:36.460978 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:36.460978 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:36.460978 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:36.460978 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:36.460978 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:36.460978 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:36.460978 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:36.460978 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:36.460978 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:36.460978 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:36.460978 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:58:36.460978 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:58:36.460978 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:58:36.460978 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:58:36.461874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:36.461874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:36.461874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:36.461874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:36.461874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:36.461874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:36.461874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:36.461874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:36.461874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:36.461874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:36.461874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:36.461874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:36.461874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:36.461874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:36.461874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:36.461874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:36.461874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:58:36.462764 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:58:36.462764 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:58:36.462764 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:36.462764 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 413/822] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:58:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:36.467943 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893a4ba-a02d-4124-9408-e487e44adf57 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:36.564249 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893a4ba-a02d-4124-9408-e487e44adf57 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:36.596789 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207640f-775d-40c9-95ad-b0e1587e5f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:36.597385 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207640f-775d-40c9-95ad-b0e1587e5f7b None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:36.597661 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207640f-775d-40c9-95ad-b0e1587e5f7b None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:36.597927 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207640f-775d-40c9-95ad-b0e1587e5f7b None admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:36.598559 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6207640f-775d-40c9-95ad-b0e1587e5f7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:36.598859 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6207640f-775d-40c9-95ad-b0e1587e5f7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:36.600250 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6207640f-775d-40c9-95ad-b0e1587e5f7b None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:36.612473 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 410/823] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:58:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:36.618180 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105957b5-d2cf-435e-9503-c4df6a89be65 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:36.677013 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:36.718874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105957b5-d2cf-435e-9503-c4df6a89be65 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:36.751098 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c5562-8a4e-4be2-abc8-226b70822bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:36.751722 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c5562-8a4e-4be2-abc8-226b70822bee None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:36.752001 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c5562-8a4e-4be2-abc8-226b70822bee None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:36.752274 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c5562-8a4e-4be2-abc8-226b70822bee None admin] PATH_INFO: `/v3/users/nova` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:36.759446 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988c5562-8a4e-4be2-abc8-226b70822bee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:36.760867 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988c5562-8a4e-4be2-abc8-226b70822bee None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:36.771452 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-988c5562-8a4e-4be2-abc8-226b70822bee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 13 12:58:36.771452 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:36.771452 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:36.771452 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:36.771452 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:36.771452 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:36.771452 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:36.771452 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:36.771452 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:36.771452 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:36.771452 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:36.771452 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:36.771452 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 13 12:58:36.771452 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 13 12:58:36.771452 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 13 12:58:36.771452 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 13 12:58:36.777755 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:36.777755 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:36.777755 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 13 12:58:36.777755 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:58:36.777755 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 13 12:58:36.777755 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 12:58:36.777755 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:36.777755 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:36.777755 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:36.777755 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:36.777755 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:36.777755 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:36.777755 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:36.777755 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:36.777755 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:36.777755 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:36.777755 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:36.777755 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:36.778891 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:36.778891 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:36.778891 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 13 12:58:36.778891 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 13 12:58:36.778891 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 13 12:58:36.778891 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 13 12:58:36.778891 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 13 12:58:36.778891 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 13 12:58:36.778891 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 13 12:58:36.778891 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:36.778891 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 414/824] 10.222.0.23 () {56 vars in 1093 bytes} [Thu Oct 13 12:58:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:58:36.780299 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207640f-775d-40c9-95ad-b0e1587e5f7b None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:36.780746 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84afb7-995f-4d12-b861-8436ddd771c3 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:36.780834 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84afb7-995f-4d12-b861-8436ddd771c3 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:36.781069 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84afb7-995f-4d12-b861-8436ddd771c3 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:36.782331 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 415/825] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:36.787361 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f5e7c-58b8-4f8c-baf0-29fa2a116611 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:36.787647 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f5e7c-58b8-4f8c-baf0-29fa2a116611 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:36.787923 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f5e7c-58b8-4f8c-baf0-29fa2a116611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:36.806314 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6f1f5e7c-58b8-4f8c-baf0-29fa2a116611 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:36.807535 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6f1f5e7c-58b8-4f8c-baf0-29fa2a116611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:36.808342 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:36.877696 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 416/826] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 13 12:58:36.882749 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207640f-775d-40c9-95ad-b0e1587e5f7b None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:36.916279 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f96cf-9378-4444-8a74-ff25bd69c77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:36.916877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f96cf-9378-4444-8a74-ff25bd69c77e None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:36.917151 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f96cf-9378-4444-8a74-ff25bd69c77e None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:36.917415 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f96cf-9378-4444-8a74-ff25bd69c77e None admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:36.918129 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f96cf-9378-4444-8a74-ff25bd69c77e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:36.918427 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f96cf-9378-4444-8a74-ff25bd69c77e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:36.919857 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f96cf-9378-4444-8a74-ff25bd69c77e None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:36.937535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 411/827] 10.222.0.23 () {56 vars in 1133 bytes} [Thu Oct 13 12:58:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:36.943403 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f5e7c-58b8-4f8c-baf0-29fa2a116611 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:36.991186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:37.038646 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f5e7c-58b8-4f8c-baf0-29fa2a116611 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:37.071899 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a6f74-2331-45b6-9ca8-25d72bdc0b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:37.072492 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a6f74-2331-45b6-9ca8-25d72bdc0b2d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:37.072772 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a6f74-2331-45b6-9ca8-25d72bdc0b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:37.073038 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a6f74-2331-45b6-9ca8-25d72bdc0b2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:37.136990 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a6f74-2331-45b6-9ca8-25d72bdc0b2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:37.138355 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a6f74-2331-45b6-9ca8-25d72bdc0b2d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:37.144940 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f96cf-9378-4444-8a74-ff25bd69c77e None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:37.185280 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:37.229360 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:37.235437 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 417/828] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 13 12:58:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 13 12:58:37.241105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a6f74-2331-45b6-9ca8-25d72bdc0b2d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:37.244930 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f96cf-9378-4444-8a74-ff25bd69c77e None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:37.277015 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd57b0-a920-4948-8bcc-3d651f620af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:37.277632 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd57b0-a920-4948-8bcc-3d651f620af0 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:37.277907 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd57b0-a920-4948-8bcc-3d651f620af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:37.278174 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dd57b0-a920-4948-8bcc-3d651f620af0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:37.288475 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dd57b0-a920-4948-8bcc-3d651f620af0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:58:37.293666 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-88dd57b0-a920-4948-8bcc-3d651f620af0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:58:37.293666 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:37.293666 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:37.293666 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:37.293666 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:37.293666 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:37.293666 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:37.293666 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:37.293666 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:37.293666 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:37.293666 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:37.293666 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:37.293666 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:58:37.293666 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:58:37.293666 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:58:37.293666 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:58:37.294606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:58:37.294606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:58:37.294606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:58:37.294606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:58:37.294606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:37.294606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:37.294606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:37.294606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:37.294606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:37.294606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:37.294606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:37.294606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:37.294606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:37.294606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:37.294606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:37.294606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:37.295679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:37.295679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:37.295679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:37.295679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:37.295679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:58:37.295679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:58:37.295679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:58:37.295679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:58:37.295679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:58:37.295679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:58:37.295679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:58:37.295679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:37.295679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 412/829] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 13 12:58:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:58:37.300594 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd57b0-a920-4948-8bcc-3d651f620af0 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:37.338044 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a6f74-2331-45b6-9ca8-25d72bdc0b2d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:37.370676 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f23cc3-7291-4f1e-a34c-6fbdbabf16a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:37.371315 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f23cc3-7291-4f1e-a34c-6fbdbabf16a6 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:37.371590 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f23cc3-7291-4f1e-a34c-6fbdbabf16a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:37.371857 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f23cc3-7291-4f1e-a34c-6fbdbabf16a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:37.381426 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f23cc3-7291-4f1e-a34c-6fbdbabf16a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:37.382848 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f23cc3-7291-4f1e-a34c-6fbdbabf16a6 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:37.396554 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-b2f23cc3-7291-4f1e-a34c-6fbdbabf16a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:58:37.396554 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:37.396554 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:37.396554 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:37.396554 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:37.396554 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:37.396554 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:37.396554 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:37.396554 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:37.396554 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:37.396554 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:37.396554 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:37.396554 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 12:58:37.396554 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 12:58:37.396554 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 12:58:37.396554 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 12:58:37.397547 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:37.397547 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:37.397547 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:37.397547 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:37.397547 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:37.397547 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:37.397547 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:37.397547 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:37.397547 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:37.397547 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:37.397547 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:37.397547 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:37.397547 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:37.397547 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:37.397547 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:37.397547 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:37.397547 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 12:58:37.398428 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 12:58:37.398428 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 12:58:37.398428 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:37.398428 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 418/830] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:58:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:58:37.398659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dd57b0-a920-4948-8bcc-3d651f620af0 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:37.403648 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f23cc3-7291-4f1e-a34c-6fbdbabf16a6 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:37.406293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:37.431728 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6ea0f-08ac-4f8c-81d5-20f898cb491f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:37.432322 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6ea0f-08ac-4f8c-81d5-20f898cb491f None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:37.432598 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6ea0f-08ac-4f8c-81d5-20f898cb491f None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:37.432867 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6ea0f-08ac-4f8c-81d5-20f898cb491f None admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:37.433523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f6ea0f-08ac-4f8c-81d5-20f898cb491f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:37.433817 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f6ea0f-08ac-4f8c-81d5-20f898cb491f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:37.435195 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f6ea0f-08ac-4f8c-81d5-20f898cb491f None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:37.443647 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 413/831] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:58:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:58:37.449394 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6ea0f-08ac-4f8c-81d5-20f898cb491f None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:37.459082 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:37.502205 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f23cc3-7291-4f1e-a34c-6fbdbabf16a6 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:37.535141 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f660ac73-64e0-4cc2-b6da-e91600d7c731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:37.535737 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660ac73-64e0-4cc2-b6da-e91600d7c731 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:37.536015 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660ac73-64e0-4cc2-b6da-e91600d7c731 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:37.536285 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660ac73-64e0-4cc2-b6da-e91600d7c731 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:37.536915 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f660ac73-64e0-4cc2-b6da-e91600d7c731 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:37.537212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f660ac73-64e0-4cc2-b6da-e91600d7c731 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:37.538552 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f660ac73-64e0-4cc2-b6da-e91600d7c731 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:37.546661 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6ea0f-08ac-4f8c-81d5-20f898cb491f None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:37.550766 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 419/832] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 12:58:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:37.556367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f660ac73-64e0-4cc2-b6da-e91600d7c731 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:37.581405 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4cf132-48e3-4711-9849-f0b114a51e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:37.582027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4cf132-48e3-4711-9849-f0b114a51e5b None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:37.582298 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4cf132-48e3-4711-9849-f0b114a51e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:37.582565 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4cf132-48e3-4711-9849-f0b114a51e5b None admin] PATH_INFO: `/v3/users/780419c52cdb4fa09e1d24c263946307` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:37.583699 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4cf132-48e3-4711-9849-f0b114a51e5b None admin] RBAC: Authorizing `identity:get_user(user_id=780419c52cdb4fa09e1d24c263946307)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:37.585122 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4cf132-48e3-4711-9849-f0b114a51e5b None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:37.587232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 414/833] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:58:37 2022] GET /identity/v3/users/780419c52cdb4fa09e1d24c263946307 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:58:37.592878 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4cf132-48e3-4711-9849-f0b114a51e5b None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:37.616333 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:37.654435 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f660ac73-64e0-4cc2-b6da-e91600d7c731 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:37.666916 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:37.688117 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409708b7-b016-4f1b-87fa-54b87b89c992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:37.688732 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409708b7-b016-4f1b-87fa-54b87b89c992 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:37.689013 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409708b7-b016-4f1b-87fa-54b87b89c992 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:37.689229 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4cf132-48e3-4711-9849-f0b114a51e5b None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:37.689323 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409708b7-b016-4f1b-87fa-54b87b89c992 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:37.696550 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409708b7-b016-4f1b-87fa-54b87b89c992 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:37.697979 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409708b7-b016-4f1b-87fa-54b87b89c992 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:37.709009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-409708b7-b016-4f1b-87fa-54b87b89c992 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:58:37.709009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:37.709009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:37.709009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:37.709009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:37.709009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:37.709009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:37.709009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:37.709009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:37.709009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:37.709009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:37.709009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:37.709009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 13 12:58:37.709009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 13 12:58:37.709009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 13 12:58:37.710097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 13 12:58:37.710097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:37.710097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:37.710097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:37.710097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:37.710097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:37.710097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:37.710097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:37.710097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:37.710097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:37.710097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:37.710097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:37.710097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:37.710097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:37.710097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:37.710097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:37.710097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:37.711212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 13 12:58:37.711212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 13 12:58:37.711212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 13 12:58:37.711212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 13 12:58:37.711212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 13 12:58:37.711212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 13 12:58:37.711212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 12:58:37.711212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:37.711212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 420/834] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 12:58:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 12:58:37.716522 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409708b7-b016-4f1b-87fa-54b87b89c992 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:37.727486 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5cc5bb-5e33-497b-84c6-b2cb1f26c0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:37.728092 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cc5bb-5e33-497b-84c6-b2cb1f26c0eb None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:37.728606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cc5bb-5e33-497b-84c6-b2cb1f26c0eb None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:37.728606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cc5bb-5e33-497b-84c6-b2cb1f26c0eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:37.793174 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5cc5bb-5e33-497b-84c6-b2cb1f26c0eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:37.794616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5cc5bb-5e33-497b-84c6-b2cb1f26c0eb None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:37.814287 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409708b7-b016-4f1b-87fa-54b87b89c992 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:37.836039 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:37.849839 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d488c-2519-4397-bd32-08a33a7fa678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:37.850430 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d488c-2519-4397-bd32-08a33a7fa678 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:37.850702 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d488c-2519-4397-bd32-08a33a7fa678 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:37.850970 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d488c-2519-4397-bd32-08a33a7fa678 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:37.851682 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d488c-2519-4397-bd32-08a33a7fa678 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:37.852016 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d488c-2519-4397-bd32-08a33a7fa678 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:37.853414 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d488c-2519-4397-bd32-08a33a7fa678 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:37.864750 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 421/835] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 13 12:58:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:37.868124 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:37.870889 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d488c-2519-4397-bd32-08a33a7fa678 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:37.892182 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 415/836] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 13 12:58:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 13 12:58:37.897898 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5cc5bb-5e33-497b-84c6-b2cb1f26c0eb None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:37.968123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d488c-2519-4397-bd32-08a33a7fa678 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:37.994031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5cc5bb-5e33-497b-84c6-b2cb1f26c0eb None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:38.000065 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3154c9-45b5-4cca-b11d-a35a6bb28686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:38.000659 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3154c9-45b5-4cca-b11d-a35a6bb28686 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:38.000937 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3154c9-45b5-4cca-b11d-a35a6bb28686 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:38.001204 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3154c9-45b5-4cca-b11d-a35a6bb28686 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:38.001921 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3154c9-45b5-4cca-b11d-a35a6bb28686 None admin] RBAC: Adding query filter params (role.id=13498f16883f4cf0b7d010ec7e4d0866, scope.project.id=02308892635844b088ecdcb5413a21a0, user.id=12700d985a724595af630c81e63f86d7) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:38.002226 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3154c9-45b5-4cca-b11d-a35a6bb28686 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:38.003670 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3154c9-45b5-4cca-b11d-a35a6bb28686 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:38.009707 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 422/837] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 13 12:58:37 2022] GET /identity/v3/role_assignments?user.id=12700d985a724595af630c81e63f86d7&scope.project.id=02308892635844b088ecdcb5413a21a0&role.id=13498f16883f4cf0b7d010ec7e4d0866 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:58:38.028388 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbaf181-0548-4239-b161-1f4c3c589506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:38.029038 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbaf181-0548-4239-b161-1f4c3c589506 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:38.029314 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbaf181-0548-4239-b161-1f4c3c589506 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:38.029583 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbaf181-0548-4239-b161-1f4c3c589506 None admin] PATH_INFO: `/v3/projects/21bac567038642bdb14df2ee5e66934c` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:38.030747 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbaf181-0548-4239-b161-1f4c3c589506 None admin] RBAC: Authorizing `identity:get_project(project_id=21bac567038642bdb14df2ee5e66934c)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:38.032353 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbaf181-0548-4239-b161-1f4c3c589506 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:38.034061 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 416/838] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 13 12:58:37 2022] GET /identity/v3/projects/21bac567038642bdb14df2ee5e66934c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:58:38.040608 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3154c9-45b5-4cca-b11d-a35a6bb28686 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:38.089614 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:38.144176 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3154c9-45b5-4cca-b11d-a35a6bb28686 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:38.177903 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68f733-cd74-42ed-9a48-5263a9519a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:38.178440 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68f733-cd74-42ed-9a48-5263a9519a44 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:38.178715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68f733-cd74-42ed-9a48-5263a9519a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:38.178985 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68f733-cd74-42ed-9a48-5263a9519a44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:38.179739 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68f733-cd74-42ed-9a48-5263a9519a44 None admin] RBAC: Adding query filter params (role.id=70af94fc6a8149f2a38f74b843e1a9f0, scope.project.id=21bac567038642bdb14df2ee5e66934c, user.id=780419c52cdb4fa09e1d24c263946307) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:38.180569 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68f733-cd74-42ed-9a48-5263a9519a44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:38.181995 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68f733-cd74-42ed-9a48-5263a9519a44 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:38.188118 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 423/839] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 13 12:58:38 2022] GET /identity/v3/role_assignments?user.id=780419c52cdb4fa09e1d24c263946307&scope.project.id=21bac567038642bdb14df2ee5e66934c&role.id=70af94fc6a8149f2a38f74b843e1a9f0 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:40.337913 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4fe449-280f-4b1d-8b02-e0b956cd5208 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:40.338417 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4fe449-280f-4b1d-8b02-e0b956cd5208 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:40.338519 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4fe449-280f-4b1d-8b02-e0b956cd5208 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:40.339819 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 417/840] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:40.345073 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf887908-3513-4798-b503-43c13094385a None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:40.345475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf887908-3513-4798-b503-43c13094385a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:40.345625 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf887908-3513-4798-b503-43c13094385a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:40.363854 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-bf887908-3513-4798-b503-43c13094385a None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:40.365032 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-bf887908-3513-4798-b503-43c13094385a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:40.427731 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 424/841] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 13 12:58:40.698214 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4fe449-280f-4b1d-8b02-e0b956cd5208 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:40.743402 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:40.795967 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4fe449-280f-4b1d-8b02-e0b956cd5208 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:40.828794 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa345eb8-da41-45b1-968e-d5b9ec1322c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:40.829401 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa345eb8-da41-45b1-968e-d5b9ec1322c6 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:40.829674 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa345eb8-da41-45b1-968e-d5b9ec1322c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:40.829937 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa345eb8-da41-45b1-968e-d5b9ec1322c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:40.831015 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa345eb8-da41-45b1-968e-d5b9ec1322c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:40.832476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa345eb8-da41-45b1-968e-d5b9ec1322c6 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:40.834015 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 418/842] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:58:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:40.840334 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf887908-3513-4798-b503-43c13094385a None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:40.878904 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:40.938144 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf887908-3513-4798-b503-43c13094385a None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:40.971338 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a88c6a-e605-4e44-ba29-d16bdccc8b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:40.971864 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a88c6a-e605-4e44-ba29-d16bdccc8b57 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:40.972145 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a88c6a-e605-4e44-ba29-d16bdccc8b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:40.972416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a88c6a-e605-4e44-ba29-d16bdccc8b57 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:40.973063 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a88c6a-e605-4e44-ba29-d16bdccc8b57 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:40.974474 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a88c6a-e605-4e44-ba29-d16bdccc8b57 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:40.994839 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 425/843] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 13 12:58:40 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:58:43.119265 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a66454-8742-46be-8af1-11d04fce2a05 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:43.119804 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a66454-8742-46be-8af1-11d04fce2a05 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:43.119804 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a66454-8742-46be-8af1-11d04fce2a05 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:43.121104 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 419/844] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:43.126112 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146951fe-8ada-42ad-ae62-b0c8e364de19 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:43.126474 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146951fe-8ada-42ad-ae62-b0c8e364de19 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:43.126661 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146951fe-8ada-42ad-ae62-b0c8e364de19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:43.144164 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-146951fe-8ada-42ad-ae62-b0c8e364de19 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:43.145336 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-146951fe-8ada-42ad-ae62-b0c8e364de19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:43.208377 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 426/845] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:58:43.473267 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a66454-8742-46be-8af1-11d04fce2a05 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:43.571708 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a66454-8742-46be-8af1-11d04fce2a05 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:43.584331 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:43.604454 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e6191-2e0c-4d24-95bd-198af517e564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:43.605066 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e6191-2e0c-4d24-95bd-198af517e564 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:43.605343 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e6191-2e0c-4d24-95bd-198af517e564 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:43.605613 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e6191-2e0c-4d24-95bd-198af517e564 None admin] PATH_INFO: `/v3/roles/5a10c87b17414f04846918d777506811` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:43.606973 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e6191-2e0c-4d24-95bd-198af517e564 None admin] RBAC: Authorizing `identity:get_role(role_id=5a10c87b17414f04846918d777506811)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:43.608357 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e6191-2e0c-4d24-95bd-198af517e564 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:43.609666 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 420/846] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:58:43 2022] GET /identity/v3/roles/5a10c87b17414f04846918d777506811 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:43.615363 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146951fe-8ada-42ad-ae62-b0c8e364de19 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:43.641224 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:43.709849 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146951fe-8ada-42ad-ae62-b0c8e364de19 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:43.741724 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1a5aa-cce7-46af-8d41-2a9c30ad7ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:43.742321 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1a5aa-cce7-46af-8d41-2a9c30ad7ec5 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:43.743108 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1a5aa-cce7-46af-8d41-2a9c30ad7ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:43.743383 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1a5aa-cce7-46af-8d41-2a9c30ad7ec5 None admin] PATH_INFO: `/v3/users/b3cbac756ece4956b9c67ea3857337d1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:43.756581 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d1a5aa-cce7-46af-8d41-2a9c30ad7ec5 None admin] RBAC: Authorizing `identity:get_user(user_id=b3cbac756ece4956b9c67ea3857337d1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:43.758026 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d1a5aa-cce7-46af-8d41-2a9c30ad7ec5 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:43.759684 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 427/847] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:58:43 2022] GET /identity/v3/users/b3cbac756ece4956b9c67ea3857337d1 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:43.765382 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e6191-2e0c-4d24-95bd-198af517e564 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:43.860069 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e6191-2e0c-4d24-95bd-198af517e564 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:43.891758 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cae5e9-9531-4931-8b07-4add11f4361f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:43.892360 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cae5e9-9531-4931-8b07-4add11f4361f None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:43.892639 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cae5e9-9531-4931-8b07-4add11f4361f None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:43.892908 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cae5e9-9531-4931-8b07-4add11f4361f None admin] PATH_INFO: `/v3/projects/21bac567038642bdb14df2ee5e66934c` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:43.893995 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cae5e9-9531-4931-8b07-4add11f4361f None admin] RBAC: Authorizing `identity:get_project(project_id=21bac567038642bdb14df2ee5e66934c)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:43.895431 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cae5e9-9531-4931-8b07-4add11f4361f None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:43.897003 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 421/848] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 13 12:58:43 2022] GET /identity/v3/projects/21bac567038642bdb14df2ee5e66934c => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:43.902706 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1a5aa-cce7-46af-8d41-2a9c30ad7ec5 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:43.982309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:43.998423 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1a5aa-cce7-46af-8d41-2a9c30ad7ec5 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:44.029486 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444b02e-ba8c-4f55-b513-fb864c396e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:44.030121 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444b02e-ba8c-4f55-b513-fb864c396e7e None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:44.030395 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444b02e-ba8c-4f55-b513-fb864c396e7e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:44.030660 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444b02e-ba8c-4f55-b513-fb864c396e7e None admin] PATH_INFO: `/v3/projects/21bac567038642bdb14df2ee5e66934c/users/b3cbac756ece4956b9c67ea3857337d1/roles/5a10c87b17414f04846918d777506811` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:44.032741 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444b02e-ba8c-4f55-b513-fb864c396e7e None admin] RBAC: Authorizing `identity:create_grant(project_id=21bac567038642bdb14df2ee5e66934c, user_id=b3cbac756ece4956b9c67ea3857337d1, role_id=5a10c87b17414f04846918d777506811)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:44.034149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444b02e-ba8c-4f55-b513-fb864c396e7e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:44.046886 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 428/849] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 13 12:58:43 2022] PUT /identity/v3/projects/21bac567038642bdb14df2ee5e66934c/users/b3cbac756ece4956b9c67ea3857337d1/roles/5a10c87b17414f04846918d777506811 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 12:58:46.164594 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35686e-9326-4eae-a6ee-b5f951dc6693 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:46.165110 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35686e-9326-4eae-a6ee-b5f951dc6693 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:46.165110 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35686e-9326-4eae-a6ee-b5f951dc6693 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:46.166388 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 422/850] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:46.171516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777bc8e-8ded-4d27-9447-1115942750af None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:46.171924 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777bc8e-8ded-4d27-9447-1115942750af None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:46.172072 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777bc8e-8ded-4d27-9447-1115942750af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:46.190462 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-3777bc8e-8ded-4d27-9447-1115942750af None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:46.191682 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3777bc8e-8ded-4d27-9447-1115942750af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:46.253850 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 429/851] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:58:46.521372 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e35686e-9326-4eae-a6ee-b5f951dc6693 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:46.571450 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:46.621110 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e35686e-9326-4eae-a6ee-b5f951dc6693 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:46.652789 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f19ba3a-3fea-42a2-9fea-bb19cbfe4d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:46.653396 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19ba3a-3fea-42a2-9fea-bb19cbfe4d31 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:46.653671 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19ba3a-3fea-42a2-9fea-bb19cbfe4d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:46.653940 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19ba3a-3fea-42a2-9fea-bb19cbfe4d31 None admin] PATH_INFO: `/v3/roles/5a10c87b17414f04846918d777506811` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:46.655315 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f19ba3a-3fea-42a2-9fea-bb19cbfe4d31 None admin] RBAC: Authorizing `identity:get_role(role_id=5a10c87b17414f04846918d777506811)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:46.656671 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f19ba3a-3fea-42a2-9fea-bb19cbfe4d31 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:46.657953 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 423/852] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:58:46 2022] GET /identity/v3/roles/5a10c87b17414f04846918d777506811 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:46.663675 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777bc8e-8ded-4d27-9447-1115942750af None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:46.735077 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:46.759543 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777bc8e-8ded-4d27-9447-1115942750af None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:46.791624 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163e026-1859-43a6-ba40-129af40db60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:46.792177 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163e026-1859-43a6-ba40-129af40db60a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:46.792457 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163e026-1859-43a6-ba40-129af40db60a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:46.792727 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163e026-1859-43a6-ba40-129af40db60a None admin] PATH_INFO: `/v3/users/b3cbac756ece4956b9c67ea3857337d1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:46.793826 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7163e026-1859-43a6-ba40-129af40db60a None admin] RBAC: Authorizing `identity:get_user(user_id=b3cbac756ece4956b9c67ea3857337d1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:46.795264 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7163e026-1859-43a6-ba40-129af40db60a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:46.796801 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 430/853] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:58:46 2022] GET /identity/v3/users/b3cbac756ece4956b9c67ea3857337d1 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:46.802318 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f19ba3a-3fea-42a2-9fea-bb19cbfe4d31 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:46.897106 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f19ba3a-3fea-42a2-9fea-bb19cbfe4d31 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:46.912825 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:46.930568 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e0e38-7ce1-4ae1-a6a0-a01b08fb018e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:46.931218 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e0e38-7ce1-4ae1-a6a0-a01b08fb018e None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:46.931498 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e0e38-7ce1-4ae1-a6a0-a01b08fb018e None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:46.931768 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e0e38-7ce1-4ae1-a6a0-a01b08fb018e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:46.996405 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5e0e38-7ce1-4ae1-a6a0-a01b08fb018e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:46.997842 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5e0e38-7ce1-4ae1-a6a0-a01b08fb018e None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:47.091864 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 424/854] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 13 12:58:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 13 12:58:47.097782 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163e026-1859-43a6-ba40-129af40db60a None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:47.191378 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163e026-1859-43a6-ba40-129af40db60a None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:47.222709 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db490f3-b7d4-4944-b938-bea298a98027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:47.223343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db490f3-b7d4-4944-b938-bea298a98027 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:47.223623 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db490f3-b7d4-4944-b938-bea298a98027 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:47.223892 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db490f3-b7d4-4944-b938-bea298a98027 None admin] PATH_INFO: `/v3/projects/21bac567038642bdb14df2ee5e66934c` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:47.224988 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db490f3-b7d4-4944-b938-bea298a98027 None admin] RBAC: Authorizing `identity:get_project(project_id=21bac567038642bdb14df2ee5e66934c)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:47.226415 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db490f3-b7d4-4944-b938-bea298a98027 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:47.228017 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 431/855] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 13 12:58:47 2022] GET /identity/v3/projects/21bac567038642bdb14df2ee5e66934c => generated 311 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:47.234057 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e0e38-7ce1-4ae1-a6a0-a01b08fb018e None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:47.251041 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:47.328337 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e0e38-7ce1-4ae1-a6a0-a01b08fb018e None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:47.359668 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00316f32-5432-4fda-bd39-2f30912bd959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:47.360196 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00316f32-5432-4fda-bd39-2f30912bd959 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:47.360470 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00316f32-5432-4fda-bd39-2f30912bd959 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:47.360736 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00316f32-5432-4fda-bd39-2f30912bd959 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:47.361445 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00316f32-5432-4fda-bd39-2f30912bd959 None admin] RBAC: Adding query filter params (role.id=5a10c87b17414f04846918d777506811, scope.project.id=21bac567038642bdb14df2ee5e66934c, user.id=b3cbac756ece4956b9c67ea3857337d1) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:47.361745 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00316f32-5432-4fda-bd39-2f30912bd959 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:47.363159 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00316f32-5432-4fda-bd39-2f30912bd959 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:47.369083 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 425/856] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 13 12:58:47 2022] GET /identity/v3/role_assignments?user.id=b3cbac756ece4956b9c67ea3857337d1&scope.project.id=21bac567038642bdb14df2ee5e66934c&role.id=5a10c87b17414f04846918d777506811 => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:49.463835 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7052455-7e5d-4c3e-babb-6da685a6af21 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:49.464382 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7052455-7e5d-4c3e-babb-6da685a6af21 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:49.464382 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7052455-7e5d-4c3e-babb-6da685a6af21 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:49.465702 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 432/857] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:49.471012 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9722c-51d7-4599-a9b3-de658deeca70 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:49.471454 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9722c-51d7-4599-a9b3-de658deeca70 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:49.471629 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9722c-51d7-4599-a9b3-de658deeca70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:49.489639 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-f9e9722c-51d7-4599-a9b3-de658deeca70 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:49.490883 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-f9e9722c-51d7-4599-a9b3-de658deeca70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:49.553586 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 426/858] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:58:49.818154 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7052455-7e5d-4c3e-babb-6da685a6af21 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:49.831101 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:49.915602 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7052455-7e5d-4c3e-babb-6da685a6af21 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:49.948756 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b20556-376a-4401-92ba-2198dff6fd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:49.949372 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b20556-376a-4401-92ba-2198dff6fd0a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:49.949651 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b20556-376a-4401-92ba-2198dff6fd0a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:49.949918 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b20556-376a-4401-92ba-2198dff6fd0a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:49.951013 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b20556-376a-4401-92ba-2198dff6fd0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:49.952463 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b20556-376a-4401-92ba-2198dff6fd0a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:49.954000 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 433/859] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:58:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:49.959844 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9722c-51d7-4599-a9b3-de658deeca70 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:49.971599 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:50.053838 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9722c-51d7-4599-a9b3-de658deeca70 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:50.086695 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a7445-3b02-40ba-9065-2b7fed287868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:50.087258 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a7445-3b02-40ba-9065-2b7fed287868 None admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:50.087538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a7445-3b02-40ba-9065-2b7fed287868 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:50.087807 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a7445-3b02-40ba-9065-2b7fed287868 None admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:50.088440 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203a7445-3b02-40ba-9065-2b7fed287868 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:50.089866 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203a7445-3b02-40ba-9065-2b7fed287868 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:50.114516 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 427/860] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Oct 13 12:58:49 2022] POST /identity/v3/projects => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:58:52.256608 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b9fe6-281b-4ac7-8688-8d8f1b20f4c8 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:52.257123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b9fe6-281b-4ac7-8688-8d8f1b20f4c8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:52.257123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b9fe6-281b-4ac7-8688-8d8f1b20f4c8 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:52.258421 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 434/861] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:52.263702 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d37469-a44a-4dc8-9826-563c38568583 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:52.264110 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d37469-a44a-4dc8-9826-563c38568583 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:52.264246 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d37469-a44a-4dc8-9826-563c38568583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:52.282082 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-b2d37469-a44a-4dc8-9826-563c38568583 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:52.283274 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-b2d37469-a44a-4dc8-9826-563c38568583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:52.337441 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:52.348355 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 428/862] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:58:52.614729 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b9fe6-281b-4ac7-8688-8d8f1b20f4c8 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:52.690292 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:52.712228 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b9fe6-281b-4ac7-8688-8d8f1b20f4c8 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:52.744499 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708bdcd8-54c2-4424-b616-8df3e75ecfde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:52.745105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bdcd8-54c2-4424-b616-8df3e75ecfde None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:52.745379 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bdcd8-54c2-4424-b616-8df3e75ecfde None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:52.745649 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bdcd8-54c2-4424-b616-8df3e75ecfde None admin] PATH_INFO: `/v3/domains/default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:52.746736 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708bdcd8-54c2-4424-b616-8df3e75ecfde None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:52.748184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708bdcd8-54c2-4424-b616-8df3e75ecfde None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:52.749698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 435/863] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 12:58:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:52.755363 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d37469-a44a-4dc8-9826-563c38568583 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:52.848613 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d37469-a44a-4dc8-9826-563c38568583 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:52.881279 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960e3ab-f33e-4b10-bf38-1a380bd8cc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:52.881805 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960e3ab-f33e-4b10-bf38-1a380bd8cc6f None admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:52.882077 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960e3ab-f33e-4b10-bf38-1a380bd8cc6f None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:52.882365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960e3ab-f33e-4b10-bf38-1a380bd8cc6f None admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:52.882995 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a960e3ab-f33e-4b10-bf38-1a380bd8cc6f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:52.884427 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a960e3ab-f33e-4b10-bf38-1a380bd8cc6f None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:52.904525 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 429/864] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 13 12:58:52 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:58:55.023810 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cd0e5-6f8f-42ad-987b-f26de65d80cb None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:55.024329 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cd0e5-6f8f-42ad-987b-f26de65d80cb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:55.024329 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cd0e5-6f8f-42ad-987b-f26de65d80cb None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:55.025702 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 436/865] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:55.030715 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504cc72-2e8c-4d53-94df-b55ac613c04c None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:55.031113 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504cc72-2e8c-4d53-94df-b55ac613c04c None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:55.031298 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504cc72-2e8c-4d53-94df-b55ac613c04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:55.049930 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-0504cc72-2e8c-4d53-94df-b55ac613c04c None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:55.051105 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-0504cc72-2e8c-4d53-94df-b55ac613c04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:55.092254 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:55.114915 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 430/866] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:58:55.380854 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79cd0e5-6f8f-42ad-987b-f26de65d80cb None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:55.476986 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79cd0e5-6f8f-42ad-987b-f26de65d80cb None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:55.509353 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0305d1-af59-4543-9ade-30f1055dfe01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:55.509958 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0305d1-af59-4543-9ade-30f1055dfe01 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:55.510237 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0305d1-af59-4543-9ade-30f1055dfe01 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:55.510511 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0305d1-af59-4543-9ade-30f1055dfe01 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:55.520009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0305d1-af59-4543-9ade-30f1055dfe01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:58:55.524694 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-5a0305d1-af59-4543-9ade-30f1055dfe01 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:58:55.524694 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:55.524694 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:55.524694 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:55.524694 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:55.524694 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:55.524694 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:55.524694 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:55.524694 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:55.524694 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:55.524694 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:55.524694 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:55.524694 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:58:55.524694 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:58:55.524694 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:58:55.524694 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:58:55.525772 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:58:55.525772 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:58:55.525772 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:58:55.525772 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:58:55.525772 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:55.525772 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:55.525772 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:55.525772 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:55.525772 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:55.525772 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:55.525772 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:55.525772 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:55.525772 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:55.525772 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:55.525772 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:55.525772 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:55.526617 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:55.526617 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:55.526617 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:55.526617 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:55.526617 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:58:55.526617 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:58:55.526617 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:58:55.526617 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:58:55.526617 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:58:55.526617 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:58:55.526617 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:58:55.526617 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:58:55.526617 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 437/867] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 13 12:58:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:55.531562 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504cc72-2e8c-4d53-94df-b55ac613c04c None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:55.626110 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504cc72-2e8c-4d53-94df-b55ac613c04c None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:55.658145 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2070920-0c33-4080-8eaf-745a260fa67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:55.658674 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2070920-0c33-4080-8eaf-745a260fa67d None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:55.658947 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2070920-0c33-4080-8eaf-745a260fa67d None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:55.659238 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2070920-0c33-4080-8eaf-745a260fa67d None admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:55.659887 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2070920-0c33-4080-8eaf-745a260fa67d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:55.660183 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2070920-0c33-4080-8eaf-745a260fa67d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:55.661525 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2070920-0c33-4080-8eaf-745a260fa67d None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:55.669150 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 431/868] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:58:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:55.674966 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0305d1-af59-4543-9ade-30f1055dfe01 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:55.684584 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:55.770187 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0305d1-af59-4543-9ade-30f1055dfe01 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:55.802675 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c5238-e3ee-46d1-ac47-8216849ecaf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:55.803322 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c5238-e3ee-46d1-ac47-8216849ecaf5 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:55.803604 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c5238-e3ee-46d1-ac47-8216849ecaf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:55.803876 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c5238-e3ee-46d1-ac47-8216849ecaf5 None admin] PATH_INFO: `/v3/users/b7b35ee661c8498b8bac0ede9475bb58` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:55.817418 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c5238-e3ee-46d1-ac47-8216849ecaf5 None admin] RBAC: Authorizing `identity:get_user(user_id=b7b35ee661c8498b8bac0ede9475bb58)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:55.818839 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c5238-e3ee-46d1-ac47-8216849ecaf5 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:55.820500 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 438/869] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:58:55 2022] GET /identity/v3/users/b7b35ee661c8498b8bac0ede9475bb58 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:55.826404 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2070920-0c33-4080-8eaf-745a260fa67d None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:55.866204 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:55.922325 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2070920-0c33-4080-8eaf-745a260fa67d None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:55.954973 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3aeac-6068-4dd2-885b-286c4fec60fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:55.955619 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3aeac-6068-4dd2-885b-286c4fec60fc None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:55.955898 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3aeac-6068-4dd2-885b-286c4fec60fc None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:55.956166 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3aeac-6068-4dd2-885b-286c4fec60fc None admin] PATH_INFO: `/v3/projects/7204ae5912c646eabb9009a93a366fe5` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:55.957262 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f3aeac-6068-4dd2-885b-286c4fec60fc None admin] RBAC: Authorizing `identity:get_project(project_id=7204ae5912c646eabb9009a93a366fe5)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:55.958687 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f3aeac-6068-4dd2-885b-286c4fec60fc None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:55.960326 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 432/870] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 13 12:58:55 2022] GET /identity/v3/projects/7204ae5912c646eabb9009a93a366fe5 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:55.966174 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c5238-e3ee-46d1-ac47-8216849ecaf5 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:56.024676 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:56.060951 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c5238-e3ee-46d1-ac47-8216849ecaf5 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:56.092868 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa5ced-6836-430b-a7c4-7dc96ee4d975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:56.093506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa5ced-6836-430b-a7c4-7dc96ee4d975 None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:56.093784 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa5ced-6836-430b-a7c4-7dc96ee4d975 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:56.094057 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa5ced-6836-430b-a7c4-7dc96ee4d975 None admin] PATH_INFO: `/v3/projects/7204ae5912c646eabb9009a93a366fe5/users/b7b35ee661c8498b8bac0ede9475bb58/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:56.095947 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa5ced-6836-430b-a7c4-7dc96ee4d975 None admin] RBAC: Authorizing `identity:create_grant(project_id=7204ae5912c646eabb9009a93a366fe5, user_id=b7b35ee661c8498b8bac0ede9475bb58, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:56.097365 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa5ced-6836-430b-a7c4-7dc96ee4d975 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:56.109720 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 439/871] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 13 12:58:55 2022] PUT /identity/v3/projects/7204ae5912c646eabb9009a93a366fe5/users/b7b35ee661c8498b8bac0ede9475bb58/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 12:58:58.217537 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7154d-3c14-4856-b551-054b126713d3 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:58.218033 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7154d-3c14-4856-b551-054b126713d3 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:58.218033 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7154d-3c14-4856-b551-054b126713d3 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:58.219388 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 433/872] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:58:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:58:58.224710 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9695e8-d188-4c24-8afb-367b460e00cb None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:58.225149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9695e8-d188-4c24-8afb-367b460e00cb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:58.225265 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9695e8-d188-4c24-8afb-367b460e00cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:58.243900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-1a9695e8-d188-4c24-8afb-367b460e00cb None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:58:58.245067 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1a9695e8-d188-4c24-8afb-367b460e00cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:58:58.308104 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 440/873] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:58:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:58:58.579474 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7154d-3c14-4856-b551-054b126713d3 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:58.676620 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7154d-3c14-4856-b551-054b126713d3 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:58.685535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:58.709768 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4932b-edcf-4fad-8010-b104fa2c02f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:58.710966 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4932b-edcf-4fad-8010-b104fa2c02f1 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:58.711272 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4932b-edcf-4fad-8010-b104fa2c02f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:58.711541 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4932b-edcf-4fad-8010-b104fa2c02f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:58.721234 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4932b-edcf-4fad-8010-b104fa2c02f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:58:58.725945 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-a5e4932b-edcf-4fad-8010-b104fa2c02f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:58:58.725945 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:58:58.725945 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:58:58.725945 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:58:58.725945 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:58:58.725945 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:58:58.725945 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:58:58.725945 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:58:58.725945 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:58:58.725945 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:58:58.725945 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:58:58.725945 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:58:58.725945 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:58:58.725945 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:58:58.725945 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:58:58.725945 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:58:58.727031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:58:58.727031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:58:58.727031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:58:58.727031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:58:58.727031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:58:58.727031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:58:58.727031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:58:58.727031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:58:58.727031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:58:58.727031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:58:58.727031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:58:58.727031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:58:58.727031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:58:58.727031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:58:58.727031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:58:58.727031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:58:58.727903 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:58:58.727903 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:58:58.727903 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:58:58.727903 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:58:58.727903 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:58:58.727903 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:58:58.727903 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:58:58.727903 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:58:58.727903 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:58:58.727903 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:58:58.727903 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:58:58.727903 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:58:58.727903 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 434/874] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 13 12:58:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:58:58.733131 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9695e8-d188-4c24-8afb-367b460e00cb None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:58.785171 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:58.827995 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9695e8-d188-4c24-8afb-367b460e00cb None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:58.860107 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceaba74-1f5a-49f8-9b9d-e0aa19eb91eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:58.860636 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceaba74-1f5a-49f8-9b9d-e0aa19eb91eb None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:58.860914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceaba74-1f5a-49f8-9b9d-e0aa19eb91eb None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:58.861184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceaba74-1f5a-49f8-9b9d-e0aa19eb91eb None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:58.861832 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceaba74-1f5a-49f8-9b9d-e0aa19eb91eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:58.862128 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceaba74-1f5a-49f8-9b9d-e0aa19eb91eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:58.863502 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceaba74-1f5a-49f8-9b9d-e0aa19eb91eb None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:58.871078 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 441/875] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:58:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:58.876848 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4932b-edcf-4fad-8010-b104fa2c02f1 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:58.973897 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4932b-edcf-4fad-8010-b104fa2c02f1 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:59.006838 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3df80-d96a-484d-a83a-72e163c297c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:59.008002 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3df80-d96a-484d-a83a-72e163c297c3 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:59.008282 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3df80-d96a-484d-a83a-72e163c297c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:59.008551 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3df80-d96a-484d-a83a-72e163c297c3 None admin] PATH_INFO: `/v3/users/b7b35ee661c8498b8bac0ede9475bb58` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:59.009664 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e3df80-d96a-484d-a83a-72e163c297c3 None admin] RBAC: Authorizing `identity:get_user(user_id=b7b35ee661c8498b8bac0ede9475bb58)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:59.011109 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e3df80-d96a-484d-a83a-72e163c297c3 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:59.012672 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 435/876] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:58:58 2022] GET /identity/v3/users/b7b35ee661c8498b8bac0ede9475bb58 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:59.018440 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceaba74-1f5a-49f8-9b9d-e0aa19eb91eb None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:59.112762 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceaba74-1f5a-49f8-9b9d-e0aa19eb91eb None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:59.128013 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:59.145730 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf03b145-96c3-447a-ba44-e5a09027ad97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:59.146317 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf03b145-96c3-447a-ba44-e5a09027ad97 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:59.146591 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf03b145-96c3-447a-ba44-e5a09027ad97 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:59.146855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf03b145-96c3-447a-ba44-e5a09027ad97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:59.209366 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf03b145-96c3-447a-ba44-e5a09027ad97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:59.210806 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf03b145-96c3-447a-ba44-e5a09027ad97 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:59.302323 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 442/877] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 13 12:58:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 13 12:58:59.308144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3df80-d96a-484d-a83a-72e163c297c3 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:59.342522 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:59.402722 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3df80-d96a-484d-a83a-72e163c297c3 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:59.434012 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac52f9-9dd6-4ed8-ad43-95cf648a89d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:59.434614 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac52f9-9dd6-4ed8-ad43-95cf648a89d3 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:59.434890 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac52f9-9dd6-4ed8-ad43-95cf648a89d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:59.435436 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac52f9-9dd6-4ed8-ad43-95cf648a89d3 None admin] PATH_INFO: `/v3/projects/7204ae5912c646eabb9009a93a366fe5` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:59.436518 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ac52f9-9dd6-4ed8-ad43-95cf648a89d3 None admin] RBAC: Authorizing `identity:get_project(project_id=7204ae5912c646eabb9009a93a366fe5)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:59.437918 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ac52f9-9dd6-4ed8-ad43-95cf648a89d3 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:59.439530 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 436/878] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 13 12:58:59 2022] GET /identity/v3/projects/7204ae5912c646eabb9009a93a366fe5 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:58:59.445938 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf03b145-96c3-447a-ba44-e5a09027ad97 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:58:59.462798 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:58:59.539397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf03b145-96c3-447a-ba44-e5a09027ad97 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:58:59.571833 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c640aef-b045-4b8a-aff1-bb68311eb5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:58:59.572416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c640aef-b045-4b8a-aff1-bb68311eb5c1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:58:59.572692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c640aef-b045-4b8a-aff1-bb68311eb5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:58:59.572959 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c640aef-b045-4b8a-aff1-bb68311eb5c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:58:59.573693 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c640aef-b045-4b8a-aff1-bb68311eb5c1 None admin] RBAC: Adding query filter params (role.id=70af94fc6a8149f2a38f74b843e1a9f0, scope.project.id=7204ae5912c646eabb9009a93a366fe5, user.id=b7b35ee661c8498b8bac0ede9475bb58) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:58:59.573996 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c640aef-b045-4b8a-aff1-bb68311eb5c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:58:59.575415 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c640aef-b045-4b8a-aff1-bb68311eb5c1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:58:59.580998 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 443/879] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 13 12:58:59 2022] GET /identity/v3/role_assignments?user.id=b7b35ee661c8498b8bac0ede9475bb58&scope.project.id=7204ae5912c646eabb9009a93a366fe5&role.id=70af94fc6a8149f2a38f74b843e1a9f0 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:59:01.698555 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2468b3e-0086-4eb2-be18-a3fd182fe973 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:59:01.699119 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2468b3e-0086-4eb2-be18-a3fd182fe973 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:59:01.699119 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2468b3e-0086-4eb2-be18-a3fd182fe973 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:59:01.700438 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 437/880] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:59:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:59:01.705704 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa59e14-583b-4cd5-a334-76568cfe750c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:59:01.706120 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa59e14-583b-4cd5-a334-76568cfe750c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:59:01.706287 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa59e14-583b-4cd5-a334-76568cfe750c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:59:01.724532 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-2aa59e14-583b-4cd5-a334-76568cfe750c None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:59:01.725712 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2aa59e14-583b-4cd5-a334-76568cfe750c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:59:01.788394 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 444/881] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:59:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:59:02.052409 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2468b3e-0086-4eb2-be18-a3fd182fe973 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:59:02.150462 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2468b3e-0086-4eb2-be18-a3fd182fe973 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:59:02.152695 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:59:02.183437 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1444d767-b75d-4729-a758-20dd2326d330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:59:02.184023 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444d767-b75d-4729-a758-20dd2326d330 None admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:59:02.184304 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444d767-b75d-4729-a758-20dd2326d330 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:59:02.184573 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1444d767-b75d-4729-a758-20dd2326d330 None admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:59:02.185484 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1444d767-b75d-4729-a758-20dd2326d330 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:59:02.186843 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1444d767-b75d-4729-a758-20dd2326d330 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:59:02.202681 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 438/882] 10.222.0.23 () {60 vars in 1138 bytes} [Thu Oct 13 12:59:02 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:59:04.353494 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbfdd7-4364-4ab8-9f47-a86f6a1c1944 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:59:04.354069 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbfdd7-4364-4ab8-9f47-a86f6a1c1944 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:59:04.354069 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbfdd7-4364-4ab8-9f47-a86f6a1c1944 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:59:04.355415 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 445/883] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:59:04.360669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83e113-e9c4-40d4-a5f3-d8bf6aa5ab5e None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:59:04.361109 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83e113-e9c4-40d4-a5f3-d8bf6aa5ab5e None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:59:04.361216 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83e113-e9c4-40d4-a5f3-d8bf6aa5ab5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:59:04.379321 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-6d83e113-e9c4-40d4-a5f3-d8bf6aa5ab5e None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:59:04.380499 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-6d83e113-e9c4-40d4-a5f3-d8bf6aa5ab5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:59:04.443847 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 439/884] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:59:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:59:04.712548 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbbfdd7-4364-4ab8-9f47-a86f6a1c1944 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:59:04.724370 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:59:04.809063 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbbfdd7-4364-4ab8-9f47-a86f6a1c1944 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:59:04.840429 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7b90b9-9d07-4747-9c25-50fa00bc5a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:59:04.841039 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b90b9-9d07-4747-9c25-50fa00bc5a30 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:59:04.841323 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b90b9-9d07-4747-9c25-50fa00bc5a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:59:04.841593 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7b90b9-9d07-4747-9c25-50fa00bc5a30 None admin] PATH_INFO: `/v3/domains/215ad58e6fae4c20a7dc1cab61d82e87` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:59:04.851996 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b90b9-9d07-4747-9c25-50fa00bc5a30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=215ad58e6fae4c20a7dc1cab61d82e87)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:59:04.853409 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7b90b9-9d07-4747-9c25-50fa00bc5a30 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:59:04.854946 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 446/885] 10.222.0.23 () {56 vars in 1183 bytes} [Thu Oct 13 12:59:04 2022] GET /identity/v3/domains/215ad58e6fae4c20a7dc1cab61d82e87 => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:59:04.860609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83e113-e9c4-40d4-a5f3-d8bf6aa5ab5e None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:59:04.949463 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:59:04.956906 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83e113-e9c4-40d4-a5f3-d8bf6aa5ab5e None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:59:04.989021 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a20a7-5d37-4716-ae93-f5c83990d585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:59:04.989563 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a20a7-5d37-4716-ae93-f5c83990d585 None admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:59:04.989843 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a20a7-5d37-4716-ae93-f5c83990d585 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:59:04.990114 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a20a7-5d37-4716-ae93-f5c83990d585 None admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:59:04.990764 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616a20a7-5d37-4716-ae93-f5c83990d585 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:59:04.992845 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616a20a7-5d37-4716-ae93-f5c83990d585 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:59:05.018855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 440/886] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Oct 13 12:59:04 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:59:07.155071 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13129973-f336-4df4-a5b1-234f2524d154 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:59:07.155624 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13129973-f336-4df4-a5b1-234f2524d154 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:59:07.155624 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13129973-f336-4df4-a5b1-234f2524d154 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:59:07.156946 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 447/887] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:59:07.162201 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44acd57-a6e9-4dd9-97ea-bc83fe1ab7a9 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:59:07.162626 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44acd57-a6e9-4dd9-97ea-bc83fe1ab7a9 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:59:07.162725 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44acd57-a6e9-4dd9-97ea-bc83fe1ab7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:59:07.181550 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-f44acd57-a6e9-4dd9-97ea-bc83fe1ab7a9 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:59:07.182731 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-f44acd57-a6e9-4dd9-97ea-bc83fe1ab7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:59:07.246625 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 441/888] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:59:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:59:07.514212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13129973-f336-4df4-a5b1-234f2524d154 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:59:07.581831 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:59:07.611941 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13129973-f336-4df4-a5b1-234f2524d154 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:59:07.644174 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a47df05-4d49-4623-89f0-b761053eac48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:59:07.644780 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47df05-4d49-4623-89f0-b761053eac48 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:59:07.645063 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47df05-4d49-4623-89f0-b761053eac48 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:59:07.645333 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47df05-4d49-4623-89f0-b761053eac48 None admin] PATH_INFO: `/v3/domains/215ad58e6fae4c20a7dc1cab61d82e87` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:59:07.646423 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a47df05-4d49-4623-89f0-b761053eac48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=215ad58e6fae4c20a7dc1cab61d82e87)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:59:07.647892 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a47df05-4d49-4623-89f0-b761053eac48 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:59:07.649421 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 448/889] 10.222.0.23 () {56 vars in 1183 bytes} [Thu Oct 13 12:59:07 2022] GET /identity/v3/domains/215ad58e6fae4c20a7dc1cab61d82e87 => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:59:07.655190 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44acd57-a6e9-4dd9-97ea-bc83fe1ab7a9 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:59:07.728460 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:59:07.751622 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44acd57-a6e9-4dd9-97ea-bc83fe1ab7a9 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:59:07.784083 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b931d258-e2e0-4eab-b49e-8595419b2459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:59:07.784620 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931d258-e2e0-4eab-b49e-8595419b2459 None admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:59:07.784897 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931d258-e2e0-4eab-b49e-8595419b2459 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:59:07.785168 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931d258-e2e0-4eab-b49e-8595419b2459 None admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:59:07.785806 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b931d258-e2e0-4eab-b49e-8595419b2459 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:59:07.787261 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b931d258-e2e0-4eab-b49e-8595419b2459 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:59:07.808947 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 442/890] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 13 12:59:07 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 12:59:09.946658 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077cec6-e6ba-44c4-bac8-ffab8219e509 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:59:09.947170 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077cec6-e6ba-44c4-bac8-ffab8219e509 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:59:09.947270 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077cec6-e6ba-44c4-bac8-ffab8219e509 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:59:09.948556 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 449/891] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:59:09.953808 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3416140-abe4-4426-835d-0e17160cce75 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:59:09.954186 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3416140-abe4-4426-835d-0e17160cce75 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:59:09.954356 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3416140-abe4-4426-835d-0e17160cce75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:59:09.972510 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-c3416140-abe4-4426-835d-0e17160cce75 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:59:09.973672 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-c3416140-abe4-4426-835d-0e17160cce75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:59:10.036338 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 443/892] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:59:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:59:10.302110 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1077cec6-e6ba-44c4-bac8-ffab8219e509 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:59:10.398795 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1077cec6-e6ba-44c4-bac8-ffab8219e509 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:59:10.430890 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb5e47-a683-4dc2-9f4e-26f288a7312e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:59:10.431529 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb5e47-a683-4dc2-9f4e-26f288a7312e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:59:10.431811 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb5e47-a683-4dc2-9f4e-26f288a7312e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:59:10.432084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb5e47-a683-4dc2-9f4e-26f288a7312e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:59:10.438769 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:59:10.442731 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb5e47-a683-4dc2-9f4e-26f288a7312e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:59:10.447616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-b1eb5e47-a683-4dc2-9f4e-26f288a7312e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:59:10.447616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:59:10.447616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:59:10.447616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:59:10.447616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:59:10.447616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:59:10.447616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:59:10.447616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:59:10.447616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:59:10.447616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:59:10.447616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:59:10.447616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:59:10.447616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:59:10.447616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:59:10.447616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:59:10.447616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:59:10.448808 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:59:10.448808 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:59:10.448808 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:59:10.448808 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:59:10.448808 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:59:10.448808 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:59:10.448808 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:59:10.448808 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:59:10.448808 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:59:10.448808 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:59:10.448808 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:59:10.448808 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 12:59:10.448808 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:59:10.448808 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:59:10.448808 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:59:10.448808 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:59:10.449872 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:59:10.449872 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:59:10.449872 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:59:10.449872 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:59:10.449872 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:59:10.449872 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:59:10.449872 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:59:10.449872 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:59:10.449872 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:59:10.449872 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:59:10.449872 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:59:10.449872 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 12:59:10.449872 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 450/893] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 13 12:59:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:59:10.454618 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3416140-abe4-4426-835d-0e17160cce75 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:59:10.512218 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:59:10.550176 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3416140-abe4-4426-835d-0e17160cce75 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:59:10.582554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c92d722-6ce8-4400-b42e-12995cd2dd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:59:10.583111 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92d722-6ce8-4400-b42e-12995cd2dd2d None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:59:10.583396 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92d722-6ce8-4400-b42e-12995cd2dd2d None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:59:10.583665 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92d722-6ce8-4400-b42e-12995cd2dd2d None admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:59:10.584321 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c92d722-6ce8-4400-b42e-12995cd2dd2d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:59:10.585146 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c92d722-6ce8-4400-b42e-12995cd2dd2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:59:10.586501 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c92d722-6ce8-4400-b42e-12995cd2dd2d None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:59:10.594139 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 444/894] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:59:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:59:10.599900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb5e47-a683-4dc2-9f4e-26f288a7312e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:59:10.695616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb5e47-a683-4dc2-9f4e-26f288a7312e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:59:10.727533 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7c7f3-6cdc-4eac-8bd9-86055d5b6f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:59:10.728139 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7c7f3-6cdc-4eac-8bd9-86055d5b6f14 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:59:10.728418 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7c7f3-6cdc-4eac-8bd9-86055d5b6f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:59:10.728690 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7c7f3-6cdc-4eac-8bd9-86055d5b6f14 None admin] PATH_INFO: `/v3/users/47385ef35cfa4ffdbc26c0d7d0bf0c79` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:59:10.740968 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b7c7f3-6cdc-4eac-8bd9-86055d5b6f14 None admin] RBAC: Authorizing `identity:get_user(user_id=47385ef35cfa4ffdbc26c0d7d0bf0c79)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:59:10.742395 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b7c7f3-6cdc-4eac-8bd9-86055d5b6f14 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:59:10.743973 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 451/895] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:59:10 2022] GET /identity/v3/users/47385ef35cfa4ffdbc26c0d7d0bf0c79 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:59:10.749508 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c92d722-6ce8-4400-b42e-12995cd2dd2d None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:59:10.844804 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c92d722-6ce8-4400-b42e-12995cd2dd2d None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:59:10.866213 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:59:10.876898 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63552a2c-eb67-47b5-8450-73f784d3c864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:59:10.877511 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63552a2c-eb67-47b5-8450-73f784d3c864 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:59:10.877791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63552a2c-eb67-47b5-8450-73f784d3c864 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:59:10.878063 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63552a2c-eb67-47b5-8450-73f784d3c864 None admin] PATH_INFO: `/v3/projects/caf24e4c01de407495475e751489ad28` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:59:10.879189 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63552a2c-eb67-47b5-8450-73f784d3c864 None admin] RBAC: Authorizing `identity:get_project(project_id=caf24e4c01de407495475e751489ad28)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:59:10.880605 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63552a2c-eb67-47b5-8450-73f784d3c864 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:59:10.882687 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 445/896] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 13 12:59:10 2022] GET /identity/v3/projects/caf24e4c01de407495475e751489ad28 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:59:10.888413 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7c7f3-6cdc-4eac-8bd9-86055d5b6f14 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:59:10.942061 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:59:10.984213 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7c7f3-6cdc-4eac-8bd9-86055d5b6f14 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:59:11.016916 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91a5d0-69f4-4889-8d48-c612dd6df484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:59:11.017557 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91a5d0-69f4-4889-8d48-c612dd6df484 None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:59:11.017837 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91a5d0-69f4-4889-8d48-c612dd6df484 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:59:11.018111 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91a5d0-69f4-4889-8d48-c612dd6df484 None admin] PATH_INFO: `/v3/projects/caf24e4c01de407495475e751489ad28/users/47385ef35cfa4ffdbc26c0d7d0bf0c79/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:59:11.020001 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db91a5d0-69f4-4889-8d48-c612dd6df484 None admin] RBAC: Authorizing `identity:create_grant(project_id=caf24e4c01de407495475e751489ad28, user_id=47385ef35cfa4ffdbc26c0d7d0bf0c79, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:59:11.021422 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db91a5d0-69f4-4889-8d48-c612dd6df484 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:59:11.033172 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 452/897] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 13 12:59:10 2022] PUT /identity/v3/projects/caf24e4c01de407495475e751489ad28/users/47385ef35cfa4ffdbc26c0d7d0bf0c79/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 12:59:13.127802 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763c41e-92a4-46c0-9551-056deea58ffe None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:59:13.128365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763c41e-92a4-46c0-9551-056deea58ffe None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:59:13.128365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763c41e-92a4-46c0-9551-056deea58ffe None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:59:13.129686 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 446/898] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 12:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 12:59:13.134801 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43feb6a-7db1-4ea4-8b6a-c8662afd72cf None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:59:13.135206 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43feb6a-7db1-4ea4-8b6a-c8662afd72cf None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:59:13.135375 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43feb6a-7db1-4ea4-8b6a-c8662afd72cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:59:13.152882 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f43feb6a-7db1-4ea4-8b6a-c8662afd72cf None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 12:59:13.154060 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f43feb6a-7db1-4ea4-8b6a-c8662afd72cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 12:59:13.216858 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 453/899] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 12:59:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 12:59:13.479681 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763c41e-92a4-46c0-9551-056deea58ffe None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:59:13.577238 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763c41e-92a4-46c0-9551-056deea58ffe None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:59:13.610219 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e274c9-c40f-4cf0-90b1-4e9b41f055a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:59:13.610830 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e274c9-c40f-4cf0-90b1-4e9b41f055a3 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:59:13.611139 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e274c9-c40f-4cf0-90b1-4e9b41f055a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:59:13.611414 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e274c9-c40f-4cf0-90b1-4e9b41f055a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:59:13.621327 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e274c9-c40f-4cf0-90b1-4e9b41f055a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 12:59:13.626031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-10e274c9-c40f-4cf0-90b1-4e9b41f055a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:59:13.626031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 12:59:13.626031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 12:59:13.626031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 12:59:13.626031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 12:59:13.626031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 12:59:13.626031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 12:59:13.626031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 12:59:13.626031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 12:59:13.626031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 12:59:13.626031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 12:59:13.626031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 12:59:13.626031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 12:59:13.626031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 12:59:13.626031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 12:59:13.626031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 12:59:13.627145 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 12:59:13.627145 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 12:59:13.627145 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 12:59:13.627145 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 12:59:13.627145 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 12:59:13.627145 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 12:59:13.627145 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 12:59:13.627145 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 12:59:13.627145 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 12:59:13.627145 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 12:59:13.627145 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 12:59:13.627145 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 12:59:13.627145 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 12:59:13.627145 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 12:59:13.627145 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 12:59:13.627145 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 12:59:13.628206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 12:59:13.628206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 12:59:13.628206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 12:59:13.628206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 12:59:13.628206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 12:59:13.628206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 12:59:13.628206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 12:59:13.628206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 12:59:13.628206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 12:59:13.628206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 12:59:13.628206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 12:59:13.628206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 12:59:13.628206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 447/900] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 13 12:59:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 12:59:13.633122 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43feb6a-7db1-4ea4-8b6a-c8662afd72cf None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:59:13.676548 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:59:13.727211 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43feb6a-7db1-4ea4-8b6a-c8662afd72cf None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:59:13.758740 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cfff3d-f969-4afb-a5fe-96f8d470c074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:59:13.759292 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfff3d-f969-4afb-a5fe-96f8d470c074 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:59:13.759566 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfff3d-f969-4afb-a5fe-96f8d470c074 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:59:13.759828 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfff3d-f969-4afb-a5fe-96f8d470c074 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:59:13.760472 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cfff3d-f969-4afb-a5fe-96f8d470c074 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:59:13.760769 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cfff3d-f969-4afb-a5fe-96f8d470c074 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:59:13.762103 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cfff3d-f969-4afb-a5fe-96f8d470c074 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:59:13.769589 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 454/901] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 12:59:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:59:13.776503 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e274c9-c40f-4cf0-90b1-4e9b41f055a3 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:59:13.827237 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:59:13.872350 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e274c9-c40f-4cf0-90b1-4e9b41f055a3 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:59:13.911879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468246a3-84e0-4c2c-a1d6-6a2cf05da818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:59:13.912498 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468246a3-84e0-4c2c-a1d6-6a2cf05da818 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:59:13.912774 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468246a3-84e0-4c2c-a1d6-6a2cf05da818 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:59:13.913042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468246a3-84e0-4c2c-a1d6-6a2cf05da818 None admin] PATH_INFO: `/v3/users/47385ef35cfa4ffdbc26c0d7d0bf0c79` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:59:13.914132 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468246a3-84e0-4c2c-a1d6-6a2cf05da818 None admin] RBAC: Authorizing `identity:get_user(user_id=47385ef35cfa4ffdbc26c0d7d0bf0c79)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:59:13.915597 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468246a3-84e0-4c2c-a1d6-6a2cf05da818 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:59:13.917145 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 448/902] 10.222.0.23 () {56 vars in 1177 bytes} [Thu Oct 13 12:59:13 2022] GET /identity/v3/users/47385ef35cfa4ffdbc26c0d7d0bf0c79 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:59:13.922923 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cfff3d-f969-4afb-a5fe-96f8d470c074 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:59:14.017869 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cfff3d-f969-4afb-a5fe-96f8d470c074 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:59:14.026005 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:59:14.050081 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a239366-365b-4722-89ab-91f771d29a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:59:14.050669 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a239366-365b-4722-89ab-91f771d29a66 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:59:14.050948 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a239366-365b-4722-89ab-91f771d29a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:59:14.051252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a239366-365b-4722-89ab-91f771d29a66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:59:14.114679 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a239366-365b-4722-89ab-91f771d29a66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:59:14.116166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a239366-365b-4722-89ab-91f771d29a66 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:59:14.207332 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 455/903] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 13 12:59:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 13 12:59:14.213697 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468246a3-84e0-4c2c-a1d6-6a2cf05da818 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:59:14.306930 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468246a3-84e0-4c2c-a1d6-6a2cf05da818 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:59:14.326286 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:59:14.340344 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac758849-8fcf-4f7c-b66a-ced0c929ae83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:59:14.340962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac758849-8fcf-4f7c-b66a-ced0c929ae83 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:59:14.341242 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac758849-8fcf-4f7c-b66a-ced0c929ae83 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:59:14.341514 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac758849-8fcf-4f7c-b66a-ced0c929ae83 None admin] PATH_INFO: `/v3/projects/caf24e4c01de407495475e751489ad28` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:59:14.342611 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac758849-8fcf-4f7c-b66a-ced0c929ae83 None admin] RBAC: Authorizing `identity:get_project(project_id=caf24e4c01de407495475e751489ad28)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:59:14.344052 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac758849-8fcf-4f7c-b66a-ced0c929ae83 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:59:14.345645 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 449/904] 10.222.0.23 () {56 vars in 1186 bytes} [Thu Oct 13 12:59:14 2022] GET /identity/v3/projects/caf24e4c01de407495475e751489ad28 => generated 361 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 12:59:14.352138 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a239366-365b-4722-89ab-91f771d29a66 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 12:59:14.361776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:59:14.445423 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a239366-365b-4722-89ab-91f771d29a66 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 12:59:14.477277 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0661428-2086-4f6c-be91-c435ce1f4740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 12:59:14.477861 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0661428-2086-4f6c-be91-c435ce1f4740 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 12:59:14.478137 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0661428-2086-4f6c-be91-c435ce1f4740 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 12:59:14.478406 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0661428-2086-4f6c-be91-c435ce1f4740 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 12:59:14.479144 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0661428-2086-4f6c-be91-c435ce1f4740 None admin] RBAC: Adding query filter params (role.id=70af94fc6a8149f2a38f74b843e1a9f0, scope.project.id=caf24e4c01de407495475e751489ad28, user.id=47385ef35cfa4ffdbc26c0d7d0bf0c79) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 12:59:14.479453 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0661428-2086-4f6c-be91-c435ce1f4740 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 12:59:14.480851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0661428-2086-4f6c-be91-c435ce1f4740 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 12:59:14.486384 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 456/905] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 13 12:59:14 2022] GET /identity/v3/role_assignments?user.id=47385ef35cfa4ffdbc26c0d7d0bf0c79&scope.project.id=caf24e4c01de407495475e751489ad28&role.id=70af94fc6a8149f2a38f74b843e1a9f0 => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 12:59:24.336289 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=6 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 12:59:24.483803 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=64 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:09.626540 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b06af-cd82-4d04-b255-d3f3f998f984 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:09.626540 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b06af-cd82-4d04-b255-d3f3f998f984 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:09.626540 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b06af-cd82-4d04-b255-d3f3f998f984 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:09.626540 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 450/906] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 13 13:00:09.633288 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72959584-8920-4028-ab48-235da1bbdad7 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:09.633793 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72959584-8920-4028-ab48-235da1bbdad7 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:09.634039 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72959584-8920-4028-ab48-235da1bbdad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:09.654697 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-72959584-8920-4028-ab48-235da1bbdad7 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:00:09.656024 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-72959584-8920-4028-ab48-235da1bbdad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:00:09.743257 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 457/907] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:00:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 13:00:10.074404 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b06af-cd82-4d04-b255-d3f3f998f984 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:10.204152 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b06af-cd82-4d04-b255-d3f3f998f984 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:10.247103 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e21f18-d18b-46e9-b516-39748bb12977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:10.247103 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e21f18-d18b-46e9-b516-39748bb12977 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:10.247418 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e21f18-d18b-46e9-b516-39748bb12977 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:10.247606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e21f18-d18b-46e9-b516-39748bb12977 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:10.259025 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e21f18-d18b-46e9-b516-39748bb12977 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:10.260514 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e21f18-d18b-46e9-b516-39748bb12977 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:10.278285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-77e21f18-d18b-46e9-b516-39748bb12977 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 13:00:10.278285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 13:00:10.278285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 13:00:10.278285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 13:00:10.278285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 13:00:10.278285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 13:00:10.278285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 13:00:10.278285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 13:00:10.278285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 13:00:10.278285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 13:00:10.278285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 13:00:10.278285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 13:00:10.278285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 13:00:10.278285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 13:00:10.278285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 13:00:10.278285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 13:00:10.279690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 13:00:10.279690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 13:00:10.279690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 13:00:10.279690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 13:00:10.279690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 13:00:10.279690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 13:00:10.279690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 13:00:10.279690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 13:00:10.279690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 13:00:10.279690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 13:00:10.279690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 13:00:10.279690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 13:00:10.279690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 13:00:10.279690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 13:00:10.279690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 13:00:10.279690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 13:00:10.279690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 13:00:10.280767 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 13:00:10.280767 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 13:00:10.280767 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 13:00:10.280767 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 451/908] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 13:00:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 13:00:10.288029 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72959584-8920-4028-ab48-235da1bbdad7 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:10.427859 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72959584-8920-4028-ab48-235da1bbdad7 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:10.470874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:10.478789 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5237122-9fbe-42ae-8087-f8e238833ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:10.479485 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5237122-9fbe-42ae-8087-f8e238833ae8 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:10.479759 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5237122-9fbe-42ae-8087-f8e238833ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:10.480088 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5237122-9fbe-42ae-8087-f8e238833ae8 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:10.480796 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5237122-9fbe-42ae-8087-f8e238833ae8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:00:10.481100 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5237122-9fbe-42ae-8087-f8e238833ae8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:10.482524 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5237122-9fbe-42ae-8087-f8e238833ae8 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:10.500257 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 458/909] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 13:00:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 13:00:10.508244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e21f18-d18b-46e9-b516-39748bb12977 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:10.579236 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:10.652892 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e21f18-d18b-46e9-b516-39748bb12977 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:10.707379 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed17ad0-8495-4ce2-9af8-0bb1761673e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:10.707379 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed17ad0-8495-4ce2-9af8-0bb1761673e1 None admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:10.707379 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed17ad0-8495-4ce2-9af8-0bb1761673e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:10.707379 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed17ad0-8495-4ce2-9af8-0bb1761673e1 None admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:10.707855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed17ad0-8495-4ce2-9af8-0bb1761673e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:10.707855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed17ad0-8495-4ce2-9af8-0bb1761673e1 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:10.734595 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 452/910] 10.222.0.23 () {60 vars in 1132 bytes} [Thu Oct 13 13:00:10 2022] POST /identity/v3/users => generated 251 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:00:13.172269 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6b26b-819d-40d2-971e-2b20397c84d0 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:13.172782 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6b26b-819d-40d2-971e-2b20397c84d0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:13.172782 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6b26b-819d-40d2-971e-2b20397c84d0 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:13.174088 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 459/911] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:00:13.179634 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4bf86a-4e89-4bf4-a0eb-42c5faf2e988 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:13.180199 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4bf86a-4e89-4bf4-a0eb-42c5faf2e988 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:13.180199 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4bf86a-4e89-4bf4-a0eb-42c5faf2e988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:13.199182 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-bd4bf86a-4e89-4bf4-a0eb-42c5faf2e988 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:00:13.200382 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-bd4bf86a-4e89-4bf4-a0eb-42c5faf2e988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:00:13.276151 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 453/912] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:00:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 13:00:13.555672 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f6b26b-819d-40d2-971e-2b20397c84d0 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:13.677863 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f6b26b-819d-40d2-971e-2b20397c84d0 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:13.715780 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8db402-6f7e-477b-a191-59f10dae94c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:13.716431 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8db402-6f7e-477b-a191-59f10dae94c6 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:13.716701 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8db402-6f7e-477b-a191-59f10dae94c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:13.716972 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8db402-6f7e-477b-a191-59f10dae94c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:13.729221 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8db402-6f7e-477b-a191-59f10dae94c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 13:00:13.733947 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-5b8db402-6f7e-477b-a191-59f10dae94c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 13 13:00:13.733947 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 13:00:13.733947 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 13:00:13.733947 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 13:00:13.733947 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 13:00:13.733947 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 13:00:13.733947 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 13:00:13.733947 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 13:00:13.733947 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 13:00:13.733947 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 13:00:13.733947 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 13:00:13.733947 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 13:00:13.733947 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 13:00:13.733947 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 13:00:13.733947 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 13:00:13.733947 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 13:00:13.734914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 13:00:13.734914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 13:00:13.734914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 13:00:13.734914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 13:00:13.734914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 13:00:13.734914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 13:00:13.734914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 13:00:13.734914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 13:00:13.734914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 13:00:13.734914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 13:00:13.734914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 13:00:13.734914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 13:00:13.734914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 13:00:13.734914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 13:00:13.734914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 13:00:13.734914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 13:00:13.736271 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 13:00:13.736271 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 13:00:13.736271 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 13:00:13.736271 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 13:00:13.736271 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 13:00:13.736271 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 13:00:13.736271 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 13:00:13.736271 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 13:00:13.736271 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 13:00:13.736271 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 13:00:13.736271 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 13 13:00:13.736271 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 13:00:13.736271 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 460/913] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Oct 13 13:00:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 13:00:13.742084 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4bf86a-4e89-4bf4-a0eb-42c5faf2e988 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:13.784835 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:13.860274 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4bf86a-4e89-4bf4-a0eb-42c5faf2e988 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:13.898470 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10385be4-cee2-482d-8e7e-1c9b851a08a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:13.899381 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10385be4-cee2-482d-8e7e-1c9b851a08a2 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:13.899845 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10385be4-cee2-482d-8e7e-1c9b851a08a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:13.900122 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10385be4-cee2-482d-8e7e-1c9b851a08a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:13.900806 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10385be4-cee2-482d-8e7e-1c9b851a08a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:00:13.901114 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10385be4-cee2-482d-8e7e-1c9b851a08a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:13.902507 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10385be4-cee2-482d-8e7e-1c9b851a08a2 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:13.920599 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 454/914] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Oct 13 13:00:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:00:13.925592 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8db402-6f7e-477b-a191-59f10dae94c6 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:13.988337 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:14.043554 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8db402-6f7e-477b-a191-59f10dae94c6 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:14.084234 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e16db-12de-4c1f-9f87-1d332b9931d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:14.084864 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e16db-12de-4c1f-9f87-1d332b9931d5 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:14.085137 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e16db-12de-4c1f-9f87-1d332b9931d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:14.085409 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e16db-12de-4c1f-9f87-1d332b9931d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:14.096626 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1e16db-12de-4c1f-9f87-1d332b9931d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:14.098732 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1e16db-12de-4c1f-9f87-1d332b9931d5 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:14.114910 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-2f1e16db-12de-4c1f-9f87-1d332b9931d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 13:00:14.114910 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 13:00:14.114910 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 13:00:14.114910 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 13:00:14.114910 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 13:00:14.114910 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 13:00:14.114910 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 13:00:14.114910 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 13:00:14.114910 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 13:00:14.114910 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 13:00:14.114910 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 13:00:14.114910 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 13:00:14.114910 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 13:00:14.114910 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 13:00:14.114910 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 13:00:14.114910 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 13:00:14.119308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 13:00:14.119308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 13:00:14.119308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 13:00:14.119308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 13:00:14.119308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 13:00:14.119308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 13:00:14.119308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 13:00:14.119308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 13:00:14.119308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 13:00:14.119308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 13:00:14.119308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 13:00:14.119308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 13:00:14.119308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 13:00:14.119308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 13:00:14.119308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 13:00:14.119308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 13:00:14.119308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 13:00:14.120260 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 13:00:14.120260 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 13:00:14.120260 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 13:00:14.120260 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 461/915] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 13:00:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 13:00:14.121674 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10385be4-cee2-482d-8e7e-1c9b851a08a2 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:14.227773 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:14.232656 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10385be4-cee2-482d-8e7e-1c9b851a08a2 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:14.270376 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ae7fa-bd83-4c44-94dc-3a8888b23706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:14.271021 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ae7fa-bd83-4c44-94dc-3a8888b23706 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:14.271431 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ae7fa-bd83-4c44-94dc-3a8888b23706 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:14.271677 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ae7fa-bd83-4c44-94dc-3a8888b23706 None admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:14.272361 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ae7fa-bd83-4c44-94dc-3a8888b23706 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:00:14.272758 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ae7fa-bd83-4c44-94dc-3a8888b23706 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:14.274290 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ae7fa-bd83-4c44-94dc-3a8888b23706 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:14.289502 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 455/916] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 13:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:00:14.295447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e16db-12de-4c1f-9f87-1d332b9931d5 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:14.404596 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e16db-12de-4c1f-9f87-1d332b9931d5 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:14.406780 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:14.443900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717974e3-1b88-4f0b-9ff2-7d429dc0d39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:14.444532 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717974e3-1b88-4f0b-9ff2-7d429dc0d39b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:14.444813 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717974e3-1b88-4f0b-9ff2-7d429dc0d39b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:14.445085 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717974e3-1b88-4f0b-9ff2-7d429dc0d39b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:14.453018 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717974e3-1b88-4f0b-9ff2-7d429dc0d39b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:14.454487 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717974e3-1b88-4f0b-9ff2-7d429dc0d39b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:14.467960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-717974e3-1b88-4f0b-9ff2-7d429dc0d39b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 13 13:00:14.467960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 13:00:14.467960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 13:00:14.467960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 13:00:14.467960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 13:00:14.467960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 13:00:14.467960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 13:00:14.467960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 13:00:14.467960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 13:00:14.467960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 13:00:14.467960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 13:00:14.467960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 13:00:14.467960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 13 13:00:14.467960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 13 13:00:14.467960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 13 13:00:14.467960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 13 13:00:14.469160 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 13:00:14.469160 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 13:00:14.469160 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 13 13:00:14.469160 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 13:00:14.469160 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 13 13:00:14.469160 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 13:00:14.469160 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 13:00:14.469160 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 13:00:14.469160 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 13:00:14.469160 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 13:00:14.469160 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 13:00:14.469160 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 13:00:14.469160 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 13:00:14.469160 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 13:00:14.469160 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 13:00:14.469160 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 13:00:14.469160 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 13:00:14.469160 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 13:00:14.470226 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 13:00:14.470226 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 13:00:14.470226 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 13 13:00:14.470226 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 13 13:00:14.470226 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 13 13:00:14.470226 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 13 13:00:14.470226 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 13 13:00:14.470226 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 13 13:00:14.470226 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 13 13:00:14.470226 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 13:00:14.474088 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 462/917] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 13:00:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 13:00:14.475320 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ae7fa-bd83-4c44-94dc-3a8888b23706 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:14.592495 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ae7fa-bd83-4c44-94dc-3a8888b23706 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:14.640460 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013f1ad-5574-481f-bba1-9003d1a1862c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:14.641074 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013f1ad-5574-481f-bba1-9003d1a1862c None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:14.641349 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013f1ad-5574-481f-bba1-9003d1a1862c None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:14.641620 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013f1ad-5574-481f-bba1-9003d1a1862c None admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:14.642374 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a013f1ad-5574-481f-bba1-9003d1a1862c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:00:14.642681 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a013f1ad-5574-481f-bba1-9003d1a1862c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:14.644712 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a013f1ad-5574-481f-bba1-9003d1a1862c None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:14.668660 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 456/918] 10.222.0.23 () {56 vars in 1143 bytes} [Thu Oct 13 13:00:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:00:14.670588 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717974e3-1b88-4f0b-9ff2-7d429dc0d39b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:14.792320 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717974e3-1b88-4f0b-9ff2-7d429dc0d39b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:14.831890 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb56de-8f35-4bf3-a27a-15d0d9287b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:14.833097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb56de-8f35-4bf3-a27a-15d0d9287b86 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:14.833645 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb56de-8f35-4bf3-a27a-15d0d9287b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:14.834215 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb56de-8f35-4bf3-a27a-15d0d9287b86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:14.847667 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb56de-8f35-4bf3-a27a-15d0d9287b86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:14.849716 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb56de-8f35-4bf3-a27a-15d0d9287b86 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:14.865537 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-a4cb56de-8f35-4bf3-a27a-15d0d9287b86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 13:00:14.865537 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 13:00:14.865537 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 13:00:14.865537 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 13:00:14.865537 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 13:00:14.865537 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 13:00:14.865537 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 13:00:14.865537 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 13:00:14.865537 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 13:00:14.865537 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 13:00:14.865537 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 13:00:14.865537 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 13:00:14.865537 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 13:00:14.865537 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 13:00:14.865537 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 13:00:14.865537 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 13:00:14.866597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 13:00:14.866597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 13:00:14.866597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 13:00:14.866597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 13:00:14.866597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 13:00:14.866597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 13:00:14.866597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 13:00:14.866597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 13:00:14.866597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 13:00:14.866597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 13:00:14.866597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 13:00:14.866597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 13:00:14.866597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 13:00:14.866597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 13:00:14.866597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 13:00:14.866597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 13:00:14.866597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 13:00:14.867622 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 13:00:14.867622 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 13:00:14.867622 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 13:00:14.873003 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 463/919] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 13:00:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 13:00:14.874097 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013f1ad-5574-481f-bba1-9003d1a1862c None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:14.895991 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:14.999080 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013f1ad-5574-481f-bba1-9003d1a1862c None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:15.038892 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bcd26c-10aa-4840-b91f-ff6d90321f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:15.039554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcd26c-10aa-4840-b91f-ff6d90321f08 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:15.039833 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcd26c-10aa-4840-b91f-ff6d90321f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:15.040117 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcd26c-10aa-4840-b91f-ff6d90321f08 None admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:15.040761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bcd26c-10aa-4840-b91f-ff6d90321f08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:00:15.041144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bcd26c-10aa-4840-b91f-ff6d90321f08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:15.042450 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bcd26c-10aa-4840-b91f-ff6d90321f08 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:15.062937 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 457/920] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 13:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 13:00:15.069504 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb56de-8f35-4bf3-a27a-15d0d9287b86 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:15.089911 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:15.197861 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb56de-8f35-4bf3-a27a-15d0d9287b86 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:15.236172 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df369532-f255-4611-b3ca-f0a04fdffa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:15.236794 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df369532-f255-4611-b3ca-f0a04fdffa0a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:15.237070 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df369532-f255-4611-b3ca-f0a04fdffa0a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:15.237918 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df369532-f255-4611-b3ca-f0a04fdffa0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:15.246345 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df369532-f255-4611-b3ca-f0a04fdffa0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:15.247880 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df369532-f255-4611-b3ca-f0a04fdffa0a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:15.259783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-df369532-f255-4611-b3ca-f0a04fdffa0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 13:00:15.259783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 13:00:15.259783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 13:00:15.259783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 13:00:15.259783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 13:00:15.259783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 13:00:15.259783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 13:00:15.259783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 13:00:15.259783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 13:00:15.259783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 13:00:15.259783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 13:00:15.259783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 13:00:15.259783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 13 13:00:15.259783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 13 13:00:15.259783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 13 13:00:15.260914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 13 13:00:15.260914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 13:00:15.260914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 13:00:15.260914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 13:00:15.260914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 13:00:15.260914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 13:00:15.260914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 13:00:15.260914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 13:00:15.260914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 13:00:15.260914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 13:00:15.260914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 13:00:15.260914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 13:00:15.260914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 13:00:15.260914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 13:00:15.260914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 13:00:15.260914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 13:00:15.260914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 13:00:15.261883 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 13 13:00:15.261883 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 13 13:00:15.261883 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 13 13:00:15.261883 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 13 13:00:15.261883 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 13 13:00:15.261883 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 13 13:00:15.261883 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 13:00:15.261883 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 13:00:15.261883 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 464/921] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 13:00:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 13:00:15.267483 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bcd26c-10aa-4840-b91f-ff6d90321f08 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:15.355524 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:15.379015 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bcd26c-10aa-4840-b91f-ff6d90321f08 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:15.415945 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc3589-a745-4a53-afd8-2d858f854e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:15.416542 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc3589-a745-4a53-afd8-2d858f854e9e None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:15.416826 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc3589-a745-4a53-afd8-2d858f854e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:15.417222 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc3589-a745-4a53-afd8-2d858f854e9e None admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:15.417924 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecc3589-a745-4a53-afd8-2d858f854e9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:00:15.418243 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecc3589-a745-4a53-afd8-2d858f854e9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:15.419732 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecc3589-a745-4a53-afd8-2d858f854e9e None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:15.431851 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 458/922] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 13 13:00:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:00:15.438516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df369532-f255-4611-b3ca-f0a04fdffa0a None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:15.520007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:15.546776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df369532-f255-4611-b3ca-f0a04fdffa0a None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:15.585380 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8851409-bb72-4d49-a1e7-fcc7e394ea7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:15.586312 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8851409-bb72-4d49-a1e7-fcc7e394ea7d None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:15.586678 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8851409-bb72-4d49-a1e7-fcc7e394ea7d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:15.587122 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8851409-bb72-4d49-a1e7-fcc7e394ea7d None admin] PATH_INFO: `/v3/projects/02308892635844b088ecdcb5413a21a0/users/1a971cdb7f9d4e6692c6326c103d7303/roles/b493ef4ee9d44f449e08251ae75a77d8` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:15.602838 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8851409-bb72-4d49-a1e7-fcc7e394ea7d None admin] RBAC: Authorizing `identity:create_grant(project_id=02308892635844b088ecdcb5413a21a0, user_id=1a971cdb7f9d4e6692c6326c103d7303, role_id=b493ef4ee9d44f449e08251ae75a77d8)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:15.604370 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8851409-bb72-4d49-a1e7-fcc7e394ea7d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:15.617978 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 465/923] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 13 13:00:15 2022] PUT /identity/v3/projects/02308892635844b088ecdcb5413a21a0/users/1a971cdb7f9d4e6692c6326c103d7303/roles/b493ef4ee9d44f449e08251ae75a77d8 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:00:17.800267 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a1feb-1bf4-4e92-bd71-81372fb68078 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:17.800790 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a1feb-1bf4-4e92-bd71-81372fb68078 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:17.800790 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a1feb-1bf4-4e92-bd71-81372fb68078 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:17.802135 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 459/924] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:00:17.807322 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa0934-1370-494f-abdc-cbc040bf311d None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:17.807764 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa0934-1370-494f-abdc-cbc040bf311d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:17.807878 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa0934-1370-494f-abdc-cbc040bf311d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:17.828705 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-47fa0934-1370-494f-abdc-cbc040bf311d None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:00:17.830249 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-47fa0934-1370-494f-abdc-cbc040bf311d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:00:17.905654 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 466/925] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:00:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 13:00:18.180524 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a1feb-1bf4-4e92-bd71-81372fb68078 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:18.286888 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a1feb-1bf4-4e92-bd71-81372fb68078 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:18.329742 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e717fe8-5cab-4f2f-9c8b-b6f127c4a329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:18.330643 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e717fe8-5cab-4f2f-9c8b-b6f127c4a329 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:18.331586 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e717fe8-5cab-4f2f-9c8b-b6f127c4a329 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:18.331859 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e717fe8-5cab-4f2f-9c8b-b6f127c4a329 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:18.340393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:18.345044 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e717fe8-5cab-4f2f-9c8b-b6f127c4a329 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 13:00:18.351190 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-2e717fe8-5cab-4f2f-9c8b-b6f127c4a329 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 13 13:00:18.351190 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 13:00:18.351190 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 13:00:18.351190 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 13:00:18.351190 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 13:00:18.351190 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 13:00:18.351190 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 13:00:18.351190 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 13:00:18.351190 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 13:00:18.351190 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 13:00:18.351190 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 13:00:18.351190 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 13:00:18.351190 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 13:00:18.351190 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 13:00:18.351190 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 13:00:18.351190 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 13:00:18.352310 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 13:00:18.352310 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 13:00:18.352310 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 13:00:18.352310 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 13:00:18.352310 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 13:00:18.352310 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 13:00:18.352310 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 13:00:18.352310 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 13:00:18.352310 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 13:00:18.352310 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 13:00:18.352310 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 13:00:18.352310 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 13:00:18.352310 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 13:00:18.352310 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 13:00:18.352310 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 13:00:18.352310 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 13:00:18.353154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 13:00:18.353154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 13:00:18.353154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 13:00:18.353154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 13:00:18.353154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 13:00:18.353154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 13:00:18.353154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 13:00:18.353154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 13:00:18.353154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 13:00:18.353154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 13:00:18.353154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 13 13:00:18.353154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 13:00:18.353154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 460/926] 10.222.0.23 () {56 vars in 1102 bytes} [Thu Oct 13 13:00:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 13:00:18.358666 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa0934-1370-494f-abdc-cbc040bf311d None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:18.418468 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:18.466298 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa0934-1370-494f-abdc-cbc040bf311d None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:18.504668 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5827e3e-2e39-448f-aa46-d48ffc8d1e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:18.505259 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5827e3e-2e39-448f-aa46-d48ffc8d1e1e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:18.505531 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5827e3e-2e39-448f-aa46-d48ffc8d1e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:18.505810 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5827e3e-2e39-448f-aa46-d48ffc8d1e1e None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:18.506510 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5827e3e-2e39-448f-aa46-d48ffc8d1e1e None admin] RBAC: Adding query filter params (name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:00:18.506858 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5827e3e-2e39-448f-aa46-d48ffc8d1e1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:18.508312 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5827e3e-2e39-448f-aa46-d48ffc8d1e1e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:18.516494 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 467/927] 10.222.0.23 () {56 vars in 1103 bytes} [Thu Oct 13 13:00:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:00:18.522989 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e717fe8-5cab-4f2f-9c8b-b6f127c4a329 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:18.639164 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e717fe8-5cab-4f2f-9c8b-b6f127c4a329 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:18.673142 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab76e6-bf51-4c4b-96b1-a72194db1ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:18.673767 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab76e6-bf51-4c4b-96b1-a72194db1ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:18.674041 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab76e6-bf51-4c4b-96b1-a72194db1ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:18.674305 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab76e6-bf51-4c4b-96b1-a72194db1ea9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:18.685868 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab76e6-bf51-4c4b-96b1-a72194db1ea9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:18.687328 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab76e6-bf51-4c4b-96b1-a72194db1ea9 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:18.704259 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-b1ab76e6-bf51-4c4b-96b1-a72194db1ea9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 13:00:18.704259 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 13:00:18.704259 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 13:00:18.704259 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 13:00:18.704259 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 13:00:18.704259 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 13:00:18.704259 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 13:00:18.704259 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 13:00:18.704259 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 13:00:18.704259 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 13:00:18.704259 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 13:00:18.704259 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 13:00:18.704259 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 13:00:18.704259 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 13:00:18.704259 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 13:00:18.704259 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 13:00:18.705410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 13:00:18.705410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 13:00:18.705410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 13:00:18.705410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 13:00:18.705410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 13:00:18.705410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 13:00:18.705410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 13:00:18.705410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 13:00:18.705410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 13:00:18.705410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 13:00:18.705410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 13:00:18.705410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 13:00:18.705410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 13:00:18.705410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 13:00:18.705410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 13:00:18.705410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 13:00:18.705410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 13:00:18.706304 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 13:00:18.706304 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 13:00:18.706304 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 13:00:18.706304 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 461/928] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 13:00:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 13:00:18.711458 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5827e3e-2e39-448f-aa46-d48ffc8d1e1e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:18.822088 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5827e3e-2e39-448f-aa46-d48ffc8d1e1e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:18.838652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:18.858159 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899be57a-1545-4914-93a2-b5187904d056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:18.858798 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899be57a-1545-4914-93a2-b5187904d056 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:18.859151 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899be57a-1545-4914-93a2-b5187904d056 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:18.859463 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899be57a-1545-4914-93a2-b5187904d056 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:18.860128 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899be57a-1545-4914-93a2-b5187904d056 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:00:18.860432 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899be57a-1545-4914-93a2-b5187904d056 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:18.861937 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899be57a-1545-4914-93a2-b5187904d056 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:18.874951 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 468/929] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 13:00:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:00:18.881492 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab76e6-bf51-4c4b-96b1-a72194db1ea9 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:18.944998 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:19.000617 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab76e6-bf51-4c4b-96b1-a72194db1ea9 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:19.036198 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab75d7b-fddc-4d1e-b888-2a464e64f03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:19.036821 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab75d7b-fddc-4d1e-b888-2a464e64f03b None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:19.037130 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab75d7b-fddc-4d1e-b888-2a464e64f03b None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:19.037370 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab75d7b-fddc-4d1e-b888-2a464e64f03b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:19.044718 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab75d7b-fddc-4d1e-b888-2a464e64f03b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:19.046150 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab75d7b-fddc-4d1e-b888-2a464e64f03b None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:19.057844 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-5ab75d7b-fddc-4d1e-b888-2a464e64f03b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 13 13:00:19.057844 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 13:00:19.057844 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 13:00:19.057844 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 13:00:19.057844 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 13:00:19.057844 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 13:00:19.057844 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 13:00:19.057844 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 13:00:19.057844 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 13:00:19.057844 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 13:00:19.057844 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 13:00:19.057844 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 13:00:19.057844 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 13 13:00:19.057844 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 13 13:00:19.057844 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 13 13:00:19.057844 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 13 13:00:19.059425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 13:00:19.059425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 13:00:19.059425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 13 13:00:19.059425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 13:00:19.059425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 13 13:00:19.059425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 13:00:19.059425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 13:00:19.059425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 13:00:19.059425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 13:00:19.059425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 13:00:19.059425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 13:00:19.059425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 13:00:19.059425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 13:00:19.059425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 13:00:19.059425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 13:00:19.059425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 13:00:19.059425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 13:00:19.059425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 13:00:19.060667 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 13:00:19.060667 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 13:00:19.060667 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 13 13:00:19.060667 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 13 13:00:19.060667 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 13 13:00:19.060667 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 13 13:00:19.060667 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 13 13:00:19.060667 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 13 13:00:19.060667 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 13 13:00:19.060667 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 13:00:19.060667 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 462/930] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 13:00:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 13:00:19.065198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899be57a-1545-4914-93a2-b5187904d056 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:19.180466 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899be57a-1545-4914-93a2-b5187904d056 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:19.217684 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5288ee55-1153-4c59-aa13-59053216dc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:19.218286 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288ee55-1153-4c59-aa13-59053216dc00 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:19.218561 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288ee55-1153-4c59-aa13-59053216dc00 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:19.218826 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288ee55-1153-4c59-aa13-59053216dc00 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:19.219585 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5288ee55-1153-4c59-aa13-59053216dc00 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:00:19.219884 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5288ee55-1153-4c59-aa13-59053216dc00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:19.221298 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5288ee55-1153-4c59-aa13-59053216dc00 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:19.237936 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 469/931] 10.222.0.23 () {56 vars in 1143 bytes} [Thu Oct 13 13:00:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:00:19.243669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab75d7b-fddc-4d1e-b888-2a464e64f03b None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:19.356748 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab75d7b-fddc-4d1e-b888-2a464e64f03b None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:19.369791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:19.398850 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c6ce90-0e08-487f-8e6c-3b84d5df7639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:19.399860 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6ce90-0e08-487f-8e6c-3b84d5df7639 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:19.400466 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6ce90-0e08-487f-8e6c-3b84d5df7639 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:19.401018 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6ce90-0e08-487f-8e6c-3b84d5df7639 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:19.487915 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c6ce90-0e08-487f-8e6c-3b84d5df7639 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:19.489537 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c6ce90-0e08-487f-8e6c-3b84d5df7639 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:19.609650 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 463/932] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 13 13:00:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 13 13:00:19.617276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5288ee55-1153-4c59-aa13-59053216dc00 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:19.654566 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:19.724723 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5288ee55-1153-4c59-aa13-59053216dc00 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:19.761745 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4051b2fb-ea0a-47cc-a686-e496a2d668a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:19.762271 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4051b2fb-ea0a-47cc-a686-e496a2d668a1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:19.762639 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4051b2fb-ea0a-47cc-a686-e496a2d668a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:19.762989 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4051b2fb-ea0a-47cc-a686-e496a2d668a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:19.774493 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4051b2fb-ea0a-47cc-a686-e496a2d668a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:19.775936 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4051b2fb-ea0a-47cc-a686-e496a2d668a1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:19.792506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-4051b2fb-ea0a-47cc-a686-e496a2d668a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 13:00:19.792506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 13:00:19.792506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 13:00:19.792506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 13:00:19.792506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 13:00:19.792506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 13:00:19.792506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 13:00:19.792506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 13:00:19.792506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 13:00:19.792506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 13:00:19.792506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 13:00:19.792506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 13:00:19.792506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 13:00:19.792506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 13:00:19.792506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 13:00:19.792506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 13:00:19.793562 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 13:00:19.793562 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 13:00:19.793562 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 13:00:19.793562 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 13:00:19.793562 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 13:00:19.793562 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 13:00:19.793562 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 13:00:19.793562 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 13:00:19.793562 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 13:00:19.793562 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 13:00:19.793562 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 13:00:19.793562 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 13:00:19.793562 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 13:00:19.793562 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 13:00:19.793562 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 13:00:19.793562 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 13:00:19.793562 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 13:00:19.794444 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 13:00:19.794444 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 13:00:19.794444 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 13:00:19.794444 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 470/933] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 13:00:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 13:00:19.800181 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c6ce90-0e08-487f-8e6c-3b84d5df7639 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:19.812647 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:19.908205 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c6ce90-0e08-487f-8e6c-3b84d5df7639 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:19.952391 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cabb52-7e6c-45d6-8b7a-d3cf0b9a4c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:19.953170 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cabb52-7e6c-45d6-8b7a-d3cf0b9a4c20 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:19.953603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cabb52-7e6c-45d6-8b7a-d3cf0b9a4c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:19.954037 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cabb52-7e6c-45d6-8b7a-d3cf0b9a4c20 None admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:19.954615 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cabb52-7e6c-45d6-8b7a-d3cf0b9a4c20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:00:19.954922 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cabb52-7e6c-45d6-8b7a-d3cf0b9a4c20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:19.956333 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cabb52-7e6c-45d6-8b7a-d3cf0b9a4c20 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:19.969075 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 464/934] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 13:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:00:19.976201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4051b2fb-ea0a-47cc-a686-e496a2d668a1 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:20.072298 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:20.088677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4051b2fb-ea0a-47cc-a686-e496a2d668a1 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:20.122782 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a52c18-3952-4150-b9a6-d50437c677bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:20.123446 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a52c18-3952-4150-b9a6-d50437c677bc None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:20.123733 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a52c18-3952-4150-b9a6-d50437c677bc None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:20.124005 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a52c18-3952-4150-b9a6-d50437c677bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:20.130642 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a52c18-3952-4150-b9a6-d50437c677bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:20.132105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a52c18-3952-4150-b9a6-d50437c677bc None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:20.143843 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-71a52c18-3952-4150-b9a6-d50437c677bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 13:00:20.143843 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 13:00:20.143843 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 13:00:20.143843 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 13:00:20.143843 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 13:00:20.143843 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 13:00:20.143843 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 13:00:20.143843 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 13:00:20.143843 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 13:00:20.143843 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 13:00:20.143843 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 13:00:20.143843 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 13:00:20.143843 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 13 13:00:20.143843 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 13 13:00:20.143843 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 13 13:00:20.144902 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 13 13:00:20.144902 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 13:00:20.144902 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 13:00:20.144902 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 13:00:20.144902 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 13:00:20.144902 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 13:00:20.144902 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 13:00:20.144902 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 13:00:20.144902 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 13:00:20.144902 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 13:00:20.144902 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 13:00:20.144902 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 13:00:20.144902 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 13:00:20.144902 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 13:00:20.144902 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 13:00:20.144902 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 13:00:20.144902 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 13:00:20.145854 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 13 13:00:20.145854 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 13 13:00:20.145854 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 13 13:00:20.145854 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 13 13:00:20.145854 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 13 13:00:20.145854 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 13 13:00:20.145854 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 13:00:20.145854 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 13:00:20.150526 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 471/935] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 13:00:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 13:00:20.151723 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cabb52-7e6c-45d6-8b7a-d3cf0b9a4c20 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:20.225289 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:20.261697 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cabb52-7e6c-45d6-8b7a-d3cf0b9a4c20 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:20.298927 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67e3dc-1855-4dd8-a58d-4668839c4abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:20.299702 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67e3dc-1855-4dd8-a58d-4668839c4abe None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:20.300011 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67e3dc-1855-4dd8-a58d-4668839c4abe None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:20.300343 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67e3dc-1855-4dd8-a58d-4668839c4abe None admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:20.301119 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f67e3dc-1855-4dd8-a58d-4668839c4abe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:00:20.301449 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f67e3dc-1855-4dd8-a58d-4668839c4abe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:20.302959 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f67e3dc-1855-4dd8-a58d-4668839c4abe None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:20.316761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 465/936] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 13 13:00:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:00:20.323716 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a52c18-3952-4150-b9a6-d50437c677bc None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:20.437415 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a52c18-3952-4150-b9a6-d50437c677bc None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:20.474972 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab5913-f6b5-48c7-938e-8e52d5716443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:20.475729 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab5913-f6b5-48c7-938e-8e52d5716443 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:20.476051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab5913-f6b5-48c7-938e-8e52d5716443 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:20.476406 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab5913-f6b5-48c7-938e-8e52d5716443 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:20.477305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fab5913-f6b5-48c7-938e-8e52d5716443 None admin] RBAC: Adding query filter params (role.id=b493ef4ee9d44f449e08251ae75a77d8, scope.project.id=02308892635844b088ecdcb5413a21a0, user.id=1a971cdb7f9d4e6692c6326c103d7303) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:00:20.477674 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fab5913-f6b5-48c7-938e-8e52d5716443 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:20.481349 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fab5913-f6b5-48c7-938e-8e52d5716443 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:20.485987 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 472/937] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 13 13:00:20 2022] GET /identity/v3/role_assignments?user.id=1a971cdb7f9d4e6692c6326c103d7303&scope.project.id=02308892635844b088ecdcb5413a21a0&role.id=b493ef4ee9d44f449e08251ae75a77d8 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 13:00:22.676161 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df70d6-063e-4490-b16c-69cd37f21a51 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:22.676756 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df70d6-063e-4490-b16c-69cd37f21a51 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:22.676756 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df70d6-063e-4490-b16c-69cd37f21a51 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:22.678192 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 466/938] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:00:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:00:22.684920 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027a93c-2c93-4841-958a-7cfad525e650 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:22.685423 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027a93c-2c93-4841-958a-7cfad525e650 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:22.685532 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027a93c-2c93-4841-958a-7cfad525e650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:22.705310 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-9027a93c-2c93-4841-958a-7cfad525e650 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:00:22.706578 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9027a93c-2c93-4841-958a-7cfad525e650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:00:22.709617 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:22.778006 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 473/939] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:00:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 13:00:23.065709 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df70d6-063e-4490-b16c-69cd37f21a51 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:23.172666 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df70d6-063e-4490-b16c-69cd37f21a51 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:23.203692 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:23.210897 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8be96a-c4ba-43ba-beb2-15db95315ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:23.211567 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8be96a-c4ba-43ba-beb2-15db95315ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:23.211848 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8be96a-c4ba-43ba-beb2-15db95315ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:23.212115 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8be96a-c4ba-43ba-beb2-15db95315ed5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:23.222608 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8be96a-c4ba-43ba-beb2-15db95315ed5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 13:00:23.227415 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-4d8be96a-c4ba-43ba-beb2-15db95315ed5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 13:00:23.227415 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 13:00:23.227415 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 13:00:23.227415 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 13:00:23.227415 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 13:00:23.227415 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 13:00:23.227415 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 13:00:23.227415 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 13:00:23.227415 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 13:00:23.227415 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 13:00:23.227415 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 13:00:23.227415 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 13:00:23.227415 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 13:00:23.227415 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 13:00:23.227415 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 13:00:23.227415 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 13:00:23.228523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 13:00:23.228523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 13:00:23.228523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 13:00:23.228523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 13:00:23.228523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 13:00:23.228523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 13:00:23.228523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 13:00:23.228523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 13:00:23.228523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 13:00:23.228523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 13:00:23.228523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 13:00:23.228523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 13:00:23.228523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 13:00:23.228523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 13:00:23.228523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 13:00:23.228523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 13:00:23.229942 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 13:00:23.229942 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 13:00:23.229942 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 13:00:23.229942 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 13:00:23.229942 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 13:00:23.229942 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 13:00:23.229942 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 13:00:23.229942 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 13:00:23.229942 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 13:00:23.229942 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 13:00:23.229942 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 13:00:23.229942 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 13:00:23.229942 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 467/940] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 13 13:00:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 13:00:23.234661 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9027a93c-2c93-4841-958a-7cfad525e650 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:23.342805 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9027a93c-2c93-4841-958a-7cfad525e650 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:23.379958 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965b796-d401-4cd9-8884-b597435e06b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:23.380565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965b796-d401-4cd9-8884-b597435e06b6 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:23.380844 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965b796-d401-4cd9-8884-b597435e06b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:23.381119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965b796-d401-4cd9-8884-b597435e06b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:23.381785 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965b796-d401-4cd9-8884-b597435e06b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:00:23.382090 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965b796-d401-4cd9-8884-b597435e06b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:23.383483 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965b796-d401-4cd9-8884-b597435e06b6 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:23.392047 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 474/941] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 13:00:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:00:23.398119 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8be96a-c4ba-43ba-beb2-15db95315ed5 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:23.504137 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8be96a-c4ba-43ba-beb2-15db95315ed5 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:23.541132 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0bcba3-bc43-4b0a-b2fc-9db84a71966c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:23.541820 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0bcba3-bc43-4b0a-b2fc-9db84a71966c None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:23.542127 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0bcba3-bc43-4b0a-b2fc-9db84a71966c None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:23.542444 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0bcba3-bc43-4b0a-b2fc-9db84a71966c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:23.555400 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0bcba3-bc43-4b0a-b2fc-9db84a71966c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:23.556908 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0bcba3-bc43-4b0a-b2fc-9db84a71966c None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:23.575232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-2b0bcba3-bc43-4b0a-b2fc-9db84a71966c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 13:00:23.575232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 13:00:23.575232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 13:00:23.575232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 13:00:23.575232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 13:00:23.575232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 13:00:23.575232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 13:00:23.575232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 13:00:23.575232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 13:00:23.575232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 13:00:23.575232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 13:00:23.575232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 13:00:23.575232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 13:00:23.575232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 13:00:23.575232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 13:00:23.575232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 13:00:23.576609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 13:00:23.576609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 13:00:23.576609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 13:00:23.576609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 13:00:23.576609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 13:00:23.576609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 13:00:23.576609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 13:00:23.576609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 13:00:23.576609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 13:00:23.576609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 13:00:23.576609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 13:00:23.576609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 13:00:23.576609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 13:00:23.576609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 13:00:23.576609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 13:00:23.576609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 13:00:23.576609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 13:00:23.577751 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 13:00:23.577751 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 13:00:23.577751 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 13:00:23.577751 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 468/942] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 13:00:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 13:00:23.582736 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965b796-d401-4cd9-8884-b597435e06b6 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:23.586060 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:23.692825 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965b796-d401-4cd9-8884-b597435e06b6 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:23.729036 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c5205a-bf67-4814-8aae-2cf306bb68ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:23.729647 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c5205a-bf67-4814-8aae-2cf306bb68ca None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:23.729929 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c5205a-bf67-4814-8aae-2cf306bb68ca None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:23.730207 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c5205a-bf67-4814-8aae-2cf306bb68ca None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:23.730846 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c5205a-bf67-4814-8aae-2cf306bb68ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:00:23.731183 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c5205a-bf67-4814-8aae-2cf306bb68ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:23.732628 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c5205a-bf67-4814-8aae-2cf306bb68ca None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:23.745876 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 475/943] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 13:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:00:23.751526 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0bcba3-bc43-4b0a-b2fc-9db84a71966c None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:23.801360 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:23.862225 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0bcba3-bc43-4b0a-b2fc-9db84a71966c None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:23.896472 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4fcd6-5796-4d2c-811d-35516c1eae4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:23.897153 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4fcd6-5796-4d2c-811d-35516c1eae4a None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:23.897435 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4fcd6-5796-4d2c-811d-35516c1eae4a None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:23.897742 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4fcd6-5796-4d2c-811d-35516c1eae4a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:23.905574 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c4fcd6-5796-4d2c-811d-35516c1eae4a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:23.907441 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c4fcd6-5796-4d2c-811d-35516c1eae4a None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:23.920614 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-71c4fcd6-5796-4d2c-811d-35516c1eae4a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 13 13:00:23.920614 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 13:00:23.920614 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 13:00:23.920614 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 13:00:23.920614 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 13:00:23.920614 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 13:00:23.920614 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 13:00:23.920614 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 13:00:23.920614 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 13:00:23.920614 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 13:00:23.920614 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 13:00:23.920614 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 13:00:23.920614 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 13 13:00:23.920614 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 13 13:00:23.920614 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 13 13:00:23.920614 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 13 13:00:23.921770 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 13:00:23.921770 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 13:00:23.921770 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 13 13:00:23.921770 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 13:00:23.921770 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 13 13:00:23.921770 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 13:00:23.921770 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 13:00:23.921770 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 13:00:23.921770 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 13:00:23.921770 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 13:00:23.921770 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 13:00:23.921770 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 13:00:23.921770 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 13:00:23.921770 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 13:00:23.921770 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 13:00:23.921770 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 13:00:23.921770 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 13:00:23.921770 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 13:00:23.922956 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 13:00:23.922956 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 13:00:23.922956 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 13 13:00:23.922956 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 13 13:00:23.922956 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 13 13:00:23.922956 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 13 13:00:23.922956 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 13 13:00:23.922956 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 13 13:00:23.922956 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 13 13:00:23.922956 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 13:00:23.929575 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 469/944] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 13:00:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 13:00:23.930744 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c5205a-bf67-4814-8aae-2cf306bb68ca None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:23.996540 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:24.040775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c5205a-bf67-4814-8aae-2cf306bb68ca None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:24.077981 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e93436-4148-4cfd-aa46-43e5a4d5b1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:24.078595 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e93436-4148-4cfd-aa46-43e5a4d5b1b6 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:24.078868 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e93436-4148-4cfd-aa46-43e5a4d5b1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:24.079272 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e93436-4148-4cfd-aa46-43e5a4d5b1b6 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:24.079968 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e93436-4148-4cfd-aa46-43e5a4d5b1b6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:00:24.080310 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e93436-4148-4cfd-aa46-43e5a4d5b1b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:24.082275 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e93436-4148-4cfd-aa46-43e5a4d5b1b6 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:24.102325 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 476/945] 10.222.0.23 () {56 vars in 1143 bytes} [Thu Oct 13 13:00:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:00:24.108286 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4fcd6-5796-4d2c-811d-35516c1eae4a None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:24.216857 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4fcd6-5796-4d2c-811d-35516c1eae4a None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:24.218554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:24.253303 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53aaa9-ad53-40f0-8ba0-e1f0e6cef918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:24.253858 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53aaa9-ad53-40f0-8ba0-e1f0e6cef918 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:24.254141 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53aaa9-ad53-40f0-8ba0-e1f0e6cef918 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:24.254415 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53aaa9-ad53-40f0-8ba0-e1f0e6cef918 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:24.265062 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c53aaa9-ad53-40f0-8ba0-e1f0e6cef918 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:24.266496 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c53aaa9-ad53-40f0-8ba0-e1f0e6cef918 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:24.281221 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-2c53aaa9-ad53-40f0-8ba0-e1f0e6cef918 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 13:00:24.281221 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 13:00:24.281221 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 13:00:24.281221 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 13:00:24.281221 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 13:00:24.281221 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 13:00:24.281221 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 13:00:24.281221 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 13:00:24.281221 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 13:00:24.281221 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 13:00:24.281221 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 13:00:24.281221 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 13:00:24.281221 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 13:00:24.281221 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 13:00:24.281221 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 13:00:24.281221 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 13:00:24.282200 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 13:00:24.282200 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 13:00:24.282200 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 13:00:24.282200 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 13:00:24.282200 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 13:00:24.282200 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 13:00:24.282200 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 13:00:24.282200 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 13:00:24.282200 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 13:00:24.282200 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 13:00:24.282200 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 13:00:24.282200 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 13:00:24.282200 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 13:00:24.282200 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 13:00:24.282200 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 13:00:24.282200 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 13:00:24.282200 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 13:00:24.283139 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 13:00:24.283139 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 13:00:24.283139 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 13:00:24.283139 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 470/946] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 13:00:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 13:00:24.288104 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e93436-4148-4cfd-aa46-43e5a4d5b1b6 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:24.396671 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e93436-4148-4cfd-aa46-43e5a4d5b1b6 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:24.420487 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:24.438111 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24bf947-3ef9-4ff3-8471-4c42a090946e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:24.438960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24bf947-3ef9-4ff3-8471-4c42a090946e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:24.439330 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24bf947-3ef9-4ff3-8471-4c42a090946e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:24.439636 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24bf947-3ef9-4ff3-8471-4c42a090946e None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:24.441372 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24bf947-3ef9-4ff3-8471-4c42a090946e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:00:24.441762 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24bf947-3ef9-4ff3-8471-4c42a090946e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:24.443491 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24bf947-3ef9-4ff3-8471-4c42a090946e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:24.458090 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 477/947] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 13:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:00:24.463776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53aaa9-ad53-40f0-8ba0-e1f0e6cef918 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:24.572522 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53aaa9-ad53-40f0-8ba0-e1f0e6cef918 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:24.615817 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b601a893-6a17-4924-8be2-60c9299dd9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:24.616467 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601a893-6a17-4924-8be2-60c9299dd9d0 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:24.616774 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601a893-6a17-4924-8be2-60c9299dd9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:24.617074 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b601a893-6a17-4924-8be2-60c9299dd9d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:24.624328 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b601a893-6a17-4924-8be2-60c9299dd9d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:24.625715 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b601a893-6a17-4924-8be2-60c9299dd9d0 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:24.638297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-b601a893-6a17-4924-8be2-60c9299dd9d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 13:00:24.638297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 13:00:24.638297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 13:00:24.638297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 13:00:24.638297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 13:00:24.638297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 13:00:24.638297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 13:00:24.638297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 13:00:24.638297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 13:00:24.638297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 13:00:24.638297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 13:00:24.638297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 13:00:24.638297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 13 13:00:24.638297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 13 13:00:24.638297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 13 13:00:24.639650 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 13 13:00:24.639650 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 13:00:24.639650 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 13:00:24.639650 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 13:00:24.639650 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 13:00:24.639650 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 13:00:24.639650 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 13:00:24.639650 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 13:00:24.639650 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 13:00:24.639650 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 13:00:24.639650 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 13:00:24.639650 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 13:00:24.639650 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 13:00:24.639650 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 13:00:24.639650 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 13:00:24.639650 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 13:00:24.639650 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 13:00:24.640996 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 13 13:00:24.640996 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 13 13:00:24.640996 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 13 13:00:24.640996 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 13 13:00:24.640996 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 13 13:00:24.640996 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 13 13:00:24.640996 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 13:00:24.640996 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 13:00:24.640996 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 471/948] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 13:00:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 13:00:24.645458 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24bf947-3ef9-4ff3-8471-4c42a090946e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:24.753243 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24bf947-3ef9-4ff3-8471-4c42a090946e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:24.787152 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff7fed-004a-4819-8c2d-3124c3276778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:24.787755 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff7fed-004a-4819-8c2d-3124c3276778 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:24.788036 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff7fed-004a-4819-8c2d-3124c3276778 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:24.788317 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff7fed-004a-4819-8c2d-3124c3276778 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:24.789008 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff7fed-004a-4819-8c2d-3124c3276778 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:00:24.789316 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff7fed-004a-4819-8c2d-3124c3276778 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:24.790792 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff7fed-004a-4819-8c2d-3124c3276778 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:24.807462 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 478/949] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 13 13:00:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 13:00:24.813904 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b601a893-6a17-4924-8be2-60c9299dd9d0 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:24.833500 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:24.930372 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b601a893-6a17-4924-8be2-60c9299dd9d0 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:24.968070 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af9824-bf8c-4320-b4cc-6296f1594a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:24.968840 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af9824-bf8c-4320-b4cc-6296f1594a5d None admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:24.969139 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af9824-bf8c-4320-b4cc-6296f1594a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:24.969415 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af9824-bf8c-4320-b4cc-6296f1594a5d None admin] PATH_INFO: `/v3/projects/02308892635844b088ecdcb5413a21a0/users/1a971cdb7f9d4e6692c6326c103d7303/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:24.972079 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10af9824-bf8c-4320-b4cc-6296f1594a5d None admin] RBAC: Authorizing `identity:create_grant(project_id=02308892635844b088ecdcb5413a21a0, user_id=1a971cdb7f9d4e6692c6326c103d7303, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:24.973625 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10af9824-bf8c-4320-b4cc-6296f1594a5d None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:24.986618 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 472/950] 10.222.0.23 () {58 vars in 1439 bytes} [Thu Oct 13 13:00:24 2022] PUT /identity/v3/projects/02308892635844b088ecdcb5413a21a0/users/1a971cdb7f9d4e6692c6326c103d7303/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:00:27.307391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5eec72-adb6-4c41-9955-38501be045fe None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:27.307933 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5eec72-adb6-4c41-9955-38501be045fe None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:27.307933 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5eec72-adb6-4c41-9955-38501be045fe None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:27.309525 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 479/951] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:00:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:00:27.314924 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ec34c-b5ce-45fe-a139-b648d7fc502c None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:27.315351 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ec34c-b5ce-45fe-a139-b648d7fc502c None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:27.315515 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ec34c-b5ce-45fe-a139-b648d7fc502c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:27.334028 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-b55ec34c-b5ce-45fe-a139-b648d7fc502c None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:00:27.335243 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-b55ec34c-b5ce-45fe-a139-b648d7fc502c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:00:27.405702 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 473/952] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:00:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 13 13:00:27.701041 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5eec72-adb6-4c41-9955-38501be045fe None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:27.748361 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:27.811322 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5eec72-adb6-4c41-9955-38501be045fe None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:27.847655 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf30842-de28-4e33-91aa-618e68a0b46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:27.848286 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf30842-de28-4e33-91aa-618e68a0b46a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:27.848568 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf30842-de28-4e33-91aa-618e68a0b46a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:27.848840 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf30842-de28-4e33-91aa-618e68a0b46a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:27.860946 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf30842-de28-4e33-91aa-618e68a0b46a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 13 13:00:27.866051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-eaf30842-de28-4e33-91aa-618e68a0b46a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 13:00:27.866051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 13:00:27.866051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 13:00:27.866051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 13:00:27.866051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 13:00:27.866051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 13:00:27.866051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 13:00:27.866051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 13:00:27.866051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 13:00:27.866051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 13:00:27.866051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 13:00:27.866051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 13:00:27.866051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 13 13:00:27.866051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 13 13:00:27.866051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 13 13:00:27.866051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 13 13:00:27.867198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 13 13:00:27.867198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 13 13:00:27.867198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 13 13:00:27.867198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 13 13:00:27.867198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 13:00:27.867198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 13:00:27.867198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 13:00:27.867198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 13:00:27.867198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 13:00:27.867198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 13:00:27.867198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 13:00:27.867198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 13:00:27.867198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 13:00:27.867198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 13:00:27.867198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 13:00:27.867198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 13:00:27.868168 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 13:00:27.868168 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 13:00:27.868168 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 13:00:27.868168 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 13:00:27.868168 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 13 13:00:27.868168 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 13 13:00:27.868168 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 13 13:00:27.868168 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 13 13:00:27.868168 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 13 13:00:27.868168 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 13 13:00:27.868168 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 13 13:00:27.868168 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 13:00:27.868168 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 480/953] 10.222.0.23 () {56 vars in 1096 bytes} [Thu Oct 13 13:00:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 13:00:27.873562 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55ec34c-b5ce-45fe-a139-b648d7fc502c None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:27.877088 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:27.987662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55ec34c-b5ce-45fe-a139-b648d7fc502c None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:28.025207 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98366039-5b72-4417-9b71-447810ac22ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:28.025811 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98366039-5b72-4417-9b71-447810ac22ef None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:28.026122 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98366039-5b72-4417-9b71-447810ac22ef None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:28.026389 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98366039-5b72-4417-9b71-447810ac22ef None admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:28.027145 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98366039-5b72-4417-9b71-447810ac22ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:00:28.027447 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98366039-5b72-4417-9b71-447810ac22ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:28.028864 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98366039-5b72-4417-9b71-447810ac22ef None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:28.037287 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 474/954] 10.222.0.23 () {56 vars in 1099 bytes} [Thu Oct 13 13:00:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:00:28.043593 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf30842-de28-4e33-91aa-618e68a0b46a None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:28.148142 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:28.151437 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf30842-de28-4e33-91aa-618e68a0b46a None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:28.188864 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13171c28-ff1d-4680-b495-dfcb995c1668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:28.189561 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13171c28-ff1d-4680-b495-dfcb995c1668 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:28.190405 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13171c28-ff1d-4680-b495-dfcb995c1668 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:28.190681 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13171c28-ff1d-4680-b495-dfcb995c1668 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:28.201522 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13171c28-ff1d-4680-b495-dfcb995c1668 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:28.203269 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13171c28-ff1d-4680-b495-dfcb995c1668 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:28.221010 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-13171c28-ff1d-4680-b495-dfcb995c1668 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 13:00:28.221010 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 13:00:28.221010 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 13:00:28.221010 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 13:00:28.221010 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 13:00:28.221010 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 13:00:28.221010 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 13:00:28.221010 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 13:00:28.221010 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 13:00:28.221010 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 13:00:28.221010 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 13:00:28.221010 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 13:00:28.221010 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 13:00:28.221010 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 13:00:28.221010 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 13:00:28.221010 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 13:00:28.222362 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 13:00:28.222362 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 13:00:28.222362 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 13:00:28.222362 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 13:00:28.222362 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 13:00:28.222362 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 13:00:28.222362 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 13:00:28.222362 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 13:00:28.222362 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 13:00:28.222362 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 13:00:28.222362 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 13:00:28.222362 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 13:00:28.222362 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 13:00:28.222362 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 13:00:28.222362 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 13:00:28.222362 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 13:00:28.222362 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 13:00:28.223701 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 13:00:28.223701 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 13:00:28.223701 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 13:00:28.223701 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 481/955] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 13:00:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 13:00:28.228318 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98366039-5b72-4417-9b71-447810ac22ef None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:28.300143 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:28.337977 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98366039-5b72-4417-9b71-447810ac22ef None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:28.382365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e78e5-f448-4e3f-bdda-e8ccc0cdf268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:28.382998 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e78e5-f448-4e3f-bdda-e8ccc0cdf268 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:28.383311 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e78e5-f448-4e3f-bdda-e8ccc0cdf268 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:28.383585 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e78e5-f448-4e3f-bdda-e8ccc0cdf268 None admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:28.384247 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e78e5-f448-4e3f-bdda-e8ccc0cdf268 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:00:28.384555 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e78e5-f448-4e3f-bdda-e8ccc0cdf268 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:28.385952 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e78e5-f448-4e3f-bdda-e8ccc0cdf268 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:28.399283 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 475/956] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 13:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 13:00:28.405597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13171c28-ff1d-4680-b495-dfcb995c1668 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:28.509489 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13171c28-ff1d-4680-b495-dfcb995c1668 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:28.543740 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f115d19-9538-4e7a-b791-28e35926b15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:28.544384 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f115d19-9538-4e7a-b791-28e35926b15b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:28.544665 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f115d19-9538-4e7a-b791-28e35926b15b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:28.544946 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f115d19-9538-4e7a-b791-28e35926b15b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:28.553869 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f115d19-9538-4e7a-b791-28e35926b15b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:28.555280 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f115d19-9538-4e7a-b791-28e35926b15b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:28.568088 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-3f115d19-9538-4e7a-b791-28e35926b15b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 13 13:00:28.568088 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 13:00:28.568088 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 13:00:28.568088 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 13:00:28.568088 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 13:00:28.568088 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 13:00:28.568088 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 13:00:28.568088 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 13:00:28.568088 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 13:00:28.568088 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 13:00:28.568088 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 13:00:28.568088 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 13:00:28.568088 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 13 13:00:28.568088 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 13 13:00:28.568088 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 13 13:00:28.568088 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 13 13:00:28.574286 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 13:00:28.574286 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 13:00:28.574286 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 13 13:00:28.574286 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 13:00:28.574286 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 13 13:00:28.574286 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 13 13:00:28.574286 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 13:00:28.574286 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 13:00:28.574286 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 13:00:28.574286 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 13:00:28.574286 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 13:00:28.574286 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 13:00:28.574286 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 13:00:28.574286 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 13:00:28.574286 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 13:00:28.574286 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 13:00:28.574286 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 13:00:28.574286 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 13:00:28.575640 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 13:00:28.575640 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 13:00:28.575640 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 13 13:00:28.575640 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 13 13:00:28.575640 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 13 13:00:28.575640 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 13 13:00:28.575640 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 13 13:00:28.575640 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 13 13:00:28.575640 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 13 13:00:28.575640 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 13:00:28.575640 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 482/957] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 13:00:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 13 13:00:28.576423 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e78e5-f448-4e3f-bdda-e8ccc0cdf268 None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:28.689152 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e78e5-f448-4e3f-bdda-e8ccc0cdf268 None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:28.716177 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:28.725960 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea947f97-e3f0-4ca7-a4d0-7ecb56e671fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:28.726573 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea947f97-e3f0-4ca7-a4d0-7ecb56e671fb None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:28.726847 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea947f97-e3f0-4ca7-a4d0-7ecb56e671fb None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:28.727150 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea947f97-e3f0-4ca7-a4d0-7ecb56e671fb None admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:28.727918 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea947f97-e3f0-4ca7-a4d0-7ecb56e671fb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:00:28.728224 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea947f97-e3f0-4ca7-a4d0-7ecb56e671fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:28.729665 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea947f97-e3f0-4ca7-a4d0-7ecb56e671fb None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:28.748857 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 476/958] 10.222.0.23 () {56 vars in 1143 bytes} [Thu Oct 13 13:00:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 13:00:28.754473 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f115d19-9538-4e7a-b791-28e35926b15b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:28.769575 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:28.855923 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f115d19-9538-4e7a-b791-28e35926b15b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:28.889455 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34728be-511b-4269-b46d-0294520f8c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:28.890061 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34728be-511b-4269-b46d-0294520f8c0d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:28.890340 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34728be-511b-4269-b46d-0294520f8c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:28.890613 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34728be-511b-4269-b46d-0294520f8c0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:28.958420 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34728be-511b-4269-b46d-0294520f8c0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:28.959885 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34728be-511b-4269-b46d-0294520f8c0d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:28.973717 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:29.056447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 483/959] 10.222.0.23 () {58 vars in 1282 bytes} [Thu Oct 13 13:00:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 13 13:00:29.062512 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea947f97-e3f0-4ca7-a4d0-7ecb56e671fb None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:29.160343 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea947f97-e3f0-4ca7-a4d0-7ecb56e671fb None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:29.191903 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e27456-5db6-45fd-a708-af6113a3e8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:29.192517 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e27456-5db6-45fd-a708-af6113a3e8ea None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:29.192795 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e27456-5db6-45fd-a708-af6113a3e8ea None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:29.193062 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e27456-5db6-45fd-a708-af6113a3e8ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:29.202464 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e27456-5db6-45fd-a708-af6113a3e8ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:29.203927 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e27456-5db6-45fd-a708-af6113a3e8ea None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:29.218951 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-04e27456-5db6-45fd-a708-af6113a3e8ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 13:00:29.218951 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 13:00:29.218951 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 13:00:29.218951 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 13:00:29.218951 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 13:00:29.218951 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 13:00:29.218951 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 13:00:29.218951 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 13:00:29.218951 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 13:00:29.218951 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 13:00:29.218951 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 13:00:29.218951 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 13:00:29.218951 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 13 13:00:29.218951 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 13 13:00:29.218951 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 13 13:00:29.218951 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 13 13:00:29.220042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 13:00:29.220042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 13:00:29.220042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 13:00:29.220042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 13:00:29.220042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 13:00:29.220042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 13:00:29.220042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 13:00:29.220042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 13:00:29.220042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 13:00:29.220042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 13:00:29.220042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 13:00:29.220042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 13:00:29.220042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 13:00:29.220042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 13:00:29.220042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 13:00:29.220042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 13:00:29.220042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 13 13:00:29.221666 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 13 13:00:29.221666 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 13 13:00:29.221666 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 13:00:29.221666 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 477/960] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 13:00:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 13:00:29.226265 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34728be-511b-4269-b46d-0294520f8c0d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:29.318343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34728be-511b-4269-b46d-0294520f8c0d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:29.331492 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:29.351881 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a87dac-60a9-4bec-8512-efb3e0ce7e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:29.352459 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a87dac-60a9-4bec-8512-efb3e0ce7e0b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:29.352734 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a87dac-60a9-4bec-8512-efb3e0ce7e0b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:29.353002 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a87dac-60a9-4bec-8512-efb3e0ce7e0b None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:29.353628 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a87dac-60a9-4bec-8512-efb3e0ce7e0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:00:29.353924 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a87dac-60a9-4bec-8512-efb3e0ce7e0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:29.355283 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a87dac-60a9-4bec-8512-efb3e0ce7e0b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:29.365796 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 484/961] 10.222.0.23 () {56 vars in 1109 bytes} [Thu Oct 13 13:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:00:29.371464 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e27456-5db6-45fd-a708-af6113a3e8ea None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:29.405625 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:29.468191 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e27456-5db6-45fd-a708-af6113a3e8ea None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:29.499242 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad6ef1-3f6c-48cb-b849-b1c929f9a8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:29.499847 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad6ef1-3f6c-48cb-b849-b1c929f9a8ba None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:29.500125 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad6ef1-3f6c-48cb-b849-b1c929f9a8ba None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:29.500390 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad6ef1-3f6c-48cb-b849-b1c929f9a8ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:29.507197 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad6ef1-3f6c-48cb-b849-b1c929f9a8ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:29.508622 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad6ef1-3f6c-48cb-b849-b1c929f9a8ba None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:29.519905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application [None req-33ad6ef1-3f6c-48cb-b849-b1c929f9a8ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 13:00:29.519905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 13:00:29.519905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 13:00:29.519905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 13:00:29.519905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 13:00:29.519905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 13:00:29.519905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 13:00:29.519905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 13:00:29.519905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 13:00:29.519905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 13:00:29.519905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 13:00:29.519905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 13:00:29.519905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 13 13:00:29.519905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 13 13:00:29.519905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 13 13:00:29.520936 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 13 13:00:29.520936 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 13:00:29.520936 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 13:00:29.520936 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 13:00:29.520936 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 13:00:29.520936 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 13:00:29.520936 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 13:00:29.520936 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 13:00:29.520936 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application with Lock( Oct 13 13:00:29.520936 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 13:00:29.520936 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._enter() Oct 13 13:00:29.520936 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 13:00:29.520936 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 13:00:29.520936 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 13:00:29.520936 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.creator() Oct 13 13:00:29.520936 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 13:00:29.520936 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application created_value = creator( Oct 13 13:00:29.521862 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 13 13:00:29.521862 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 13 13:00:29.521862 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 13 13:00:29.521862 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 13 13:00:29.521862 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 13 13:00:29.521862 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 13 13:00:29.521862 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 13 13:00:29.521862 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: ERROR keystone.server.flask.application Oct 13 13:00:29.521862 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 478/962] 10.222.0.23 () {56 vars in 1111 bytes} [Thu Oct 13 13:00:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 13:00:29.526865 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a87dac-60a9-4bec-8512-efb3e0ce7e0b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:29.627982 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a87dac-60a9-4bec-8512-efb3e0ce7e0b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:29.660320 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f28c981-4183-4b59-a09d-bd7ea43450c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:29.660911 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28c981-4183-4b59-a09d-bd7ea43450c3 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:29.661190 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28c981-4183-4b59-a09d-bd7ea43450c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:29.661459 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28c981-4183-4b59-a09d-bd7ea43450c3 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:29.662138 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f28c981-4183-4b59-a09d-bd7ea43450c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:00:29.662439 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f28c981-4183-4b59-a09d-bd7ea43450c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:29.663869 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f28c981-4183-4b59-a09d-bd7ea43450c3 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:29.675261 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 485/963] 10.222.0.23 () {56 vars in 1148 bytes} [Thu Oct 13 13:00:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:00:29.681285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad6ef1-3f6c-48cb-b849-b1c929f9a8ba None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:29.772476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:29.778464 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad6ef1-3f6c-48cb-b849-b1c929f9a8ba None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:29.810426 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710be5d-dee5-4eaa-8776-ebe5a0a75403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:29.811013 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710be5d-dee5-4eaa-8776-ebe5a0a75403 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:29.811314 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710be5d-dee5-4eaa-8776-ebe5a0a75403 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:29.811587 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710be5d-dee5-4eaa-8776-ebe5a0a75403 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:29.812301 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7710be5d-dee5-4eaa-8776-ebe5a0a75403 None admin] RBAC: Adding query filter params (role.id=70af94fc6a8149f2a38f74b843e1a9f0, scope.project.id=02308892635844b088ecdcb5413a21a0, user.id=1a971cdb7f9d4e6692c6326c103d7303) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:00:29.812610 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7710be5d-dee5-4eaa-8776-ebe5a0a75403 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:29.814027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7710be5d-dee5-4eaa-8776-ebe5a0a75403 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:29.820107 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 479/964] 10.222.0.23 () {56 vars in 1374 bytes} [Thu Oct 13 13:00:29 2022] GET /identity/v3/role_assignments?user.id=1a971cdb7f9d4e6692c6326c103d7303&scope.project.id=02308892635844b088ecdcb5413a21a0&role.id=70af94fc6a8149f2a38f74b843e1a9f0 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:00:31.947714 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ba0e2-dca6-4eb6-aed6-67fe9b361eea None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:31.948221 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ba0e2-dca6-4eb6-aed6-67fe9b361eea None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:31.948341 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ba0e2-dca6-4eb6-aed6-67fe9b361eea None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:31.949613 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 486/965] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:00:31.954930 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ca184-2c6b-4d5a-9c61-8dac1379b585 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:31.955940 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ca184-2c6b-4d5a-9c61-8dac1379b585 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:31.956176 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ca184-2c6b-4d5a-9c61-8dac1379b585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:31.973902 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-089ca184-2c6b-4d5a-9c61-8dac1379b585 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:00:31.975105 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-089ca184-2c6b-4d5a-9c61-8dac1379b585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:00:32.038229 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 480/966] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:00:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 13:00:32.304772 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ba0e2-dca6-4eb6-aed6-67fe9b361eea None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:32.335072 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:32.402588 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595ba0e2-dca6-4eb6-aed6-67fe9b361eea None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:32.435148 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fbc0e6-72e0-49b6-87d4-f7aae74e604d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:32.435760 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbc0e6-72e0-49b6-87d4-f7aae74e604d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:32.436036 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbc0e6-72e0-49b6-87d4-f7aae74e604d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:32.436305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbc0e6-72e0-49b6-87d4-f7aae74e604d None admin] PATH_INFO: `/v3/services/placement` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:32.436877 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fbc0e6-72e0-49b6-87d4-f7aae74e604d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:32.438224 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fbc0e6-72e0-49b6-87d4-f7aae74e604d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:32.447129 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-e7fbc0e6-72e0-49b6-87d4-f7aae74e604d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 13 13:00:32.447129 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 13:00:32.447129 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 13 13:00:32.447129 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 13 13:00:32.447129 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 13 13:00:32.447129 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 13 13:00:32.447129 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 13 13:00:32.447129 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 13:00:32.447129 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 13 13:00:32.447129 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 13:00:32.447129 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 13:00:32.447129 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 13:00:32.447129 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 13:00:32.447129 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 13:00:32.447129 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 13:00:32.447129 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 13:00:32.447129 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 13:00:32.448326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 13:00:32.448326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 13:00:32.448326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 13:00:32.448326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 13:00:32.448326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 13:00:32.448326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 13:00:32.448326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 13 13:00:32.448326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 13 13:00:32.448326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 13 13:00:32.448326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 13 13:00:32.448326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 13:00:32.448326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 13:00:32.448326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 13:00:32.448326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 13:00:32.448326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 13:00:32.448326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 13:00:32.449181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 13:00:32.449181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 13:00:32.449181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 13:00:32.449181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 13:00:32.449181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 13:00:32.449181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 13:00:32.449181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 13:00:32.449181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 13:00:32.449181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 13:00:32.449181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 13:00:32.449181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 13 13:00:32.449181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 13:00:32.449181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 13 13:00:32.449181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 13:00:32.449181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 487/967] 10.222.0.23 () {56 vars in 1117 bytes} [Thu Oct 13 13:00:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 13:00:32.454086 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ca184-2c6b-4d5a-9c61-8dac1379b585 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:32.538745 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:32.548304 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ca184-2c6b-4d5a-9c61-8dac1379b585 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:32.583763 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed337ca-656d-4fda-abd5-f1fb534e1ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:32.584532 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed337ca-656d-4fda-abd5-f1fb534e1ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:32.584810 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed337ca-656d-4fda-abd5-f1fb534e1ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:32.585307 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed337ca-656d-4fda-abd5-f1fb534e1ec8 None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:32.586037 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed337ca-656d-4fda-abd5-f1fb534e1ec8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:00:32.586345 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed337ca-656d-4fda-abd5-f1fb534e1ec8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:32.588100 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed337ca-656d-4fda-abd5-f1fb534e1ec8 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:32.594214 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 481/968] 10.222.0.23 () {56 vars in 1116 bytes} [Thu Oct 13 13:00:32 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:00:32.600932 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fbc0e6-72e0-49b6-87d4-f7aae74e604d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:32.698875 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:32.704683 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fbc0e6-72e0-49b6-87d4-f7aae74e604d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:32.736004 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae67ee-1224-4cf4-9e49-00467ee6d74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:32.737198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae67ee-1224-4cf4-9e49-00467ee6d74e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:32.737478 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae67ee-1224-4cf4-9e49-00467ee6d74e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:32.737746 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae67ee-1224-4cf4-9e49-00467ee6d74e None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:32.738361 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae67ee-1224-4cf4-9e49-00467ee6d74e None admin] RBAC: Adding query filter params (type=placement) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:00:32.738658 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae67ee-1224-4cf4-9e49-00467ee6d74e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:32.740036 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae67ee-1224-4cf4-9e49-00467ee6d74e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:32.744213 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 488/969] 10.222.0.23 () {56 vars in 1116 bytes} [Thu Oct 13 13:00:32 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 13:00:34.855425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d99e6-dc31-42a0-8883-46aa9f4255a0 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:34.855975 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d99e6-dc31-42a0-8883-46aa9f4255a0 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:34.855975 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d99e6-dc31-42a0-8883-46aa9f4255a0 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:34.857303 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 482/970] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:00:34.863684 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995a047-47b7-47bb-98f6-dfa9006c1bf9 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:34.864169 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995a047-47b7-47bb-98f6-dfa9006c1bf9 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:34.864273 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995a047-47b7-47bb-98f6-dfa9006c1bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:34.881912 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c995a047-47b7-47bb-98f6-dfa9006c1bf9 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:00:34.883122 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c995a047-47b7-47bb-98f6-dfa9006c1bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:00:34.947669 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 489/971] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:00:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 13:00:35.215385 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d99e6-dc31-42a0-8883-46aa9f4255a0 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:35.311873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d99e6-dc31-42a0-8883-46aa9f4255a0 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:35.344849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2164db48-00ac-49e5-9d85-092836792a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:35.345443 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164db48-00ac-49e5-9d85-092836792a74 None admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:35.346277 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164db48-00ac-49e5-9d85-092836792a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:35.346547 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164db48-00ac-49e5-9d85-092836792a74 None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:35.347228 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2164db48-00ac-49e5-9d85-092836792a74 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:35.348577 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2164db48-00ac-49e5-9d85-092836792a74 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:35.362145 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 483/972] 10.222.0.23 () {60 vars in 1141 bytes} [Thu Oct 13 13:00:35 2022] POST /identity/v3/services => generated 246 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:00:37.475513 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf498c-07cc-4fae-8386-8dde2bc6eacf None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:37.476078 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf498c-07cc-4fae-8386-8dde2bc6eacf None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:37.476078 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf498c-07cc-4fae-8386-8dde2bc6eacf None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:37.477408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 490/973] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:00:37.482589 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e649299-ef34-4aa6-a8ab-252e53e7e42c None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:37.483006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e649299-ef34-4aa6-a8ab-252e53e7e42c None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:37.483140 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e649299-ef34-4aa6-a8ab-252e53e7e42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:37.501298 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-0e649299-ef34-4aa6-a8ab-252e53e7e42c None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:00:37.502461 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-0e649299-ef34-4aa6-a8ab-252e53e7e42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:00:37.511469 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:37.573814 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 484/974] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:00:37 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 13:00:37.851956 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf498c-07cc-4fae-8386-8dde2bc6eacf None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:37.939036 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:37.949999 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf498c-07cc-4fae-8386-8dde2bc6eacf None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:37.982407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e32946-6efe-4f1c-8d45-8aeedf806323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:37.983025 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e32946-6efe-4f1c-8d45-8aeedf806323 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:37.983337 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e32946-6efe-4f1c-8d45-8aeedf806323 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:37.983609 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e32946-6efe-4f1c-8d45-8aeedf806323 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:37.984190 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e32946-6efe-4f1c-8d45-8aeedf806323 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:37.985545 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e32946-6efe-4f1c-8d45-8aeedf806323 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:37.993977 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-68e32946-6efe-4f1c-8d45-8aeedf806323 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 13 13:00:37.993977 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 13:00:37.993977 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 13 13:00:37.993977 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 13 13:00:37.993977 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 13 13:00:37.993977 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 13 13:00:37.993977 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 13 13:00:37.993977 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 13:00:37.993977 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 13 13:00:37.993977 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 13:00:37.993977 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 13:00:37.993977 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 13:00:37.993977 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 13:00:37.993977 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 13:00:37.993977 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 13:00:37.993977 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 13:00:37.993977 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 13:00:37.995300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 13:00:37.995300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 13:00:37.995300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 13:00:37.995300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 13:00:37.995300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 13:00:37.995300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 13:00:37.995300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 13 13:00:37.995300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 13 13:00:37.995300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 13 13:00:37.995300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 13 13:00:37.995300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 13:00:37.995300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 13:00:37.995300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 13:00:37.995300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 13:00:37.995300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 13:00:37.995300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 13:00:37.996233 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 13:00:37.996233 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 13:00:37.996233 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 13:00:37.996233 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 13:00:37.996233 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 13:00:37.996233 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 13:00:37.996233 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 13:00:37.996233 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 13:00:37.996233 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 13:00:37.996233 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 13:00:37.996233 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 13 13:00:37.996233 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 13:00:37.996233 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 13 13:00:37.996233 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 13:00:37.996233 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 491/975] 10.222.0.23 () {56 vars in 1117 bytes} [Thu Oct 13 13:00:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 13:00:38.000984 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e649299-ef34-4aa6-a8ab-252e53e7e42c None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:38.095932 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e649299-ef34-4aa6-a8ab-252e53e7e42c None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:38.128396 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db26ff5-6ec5-4c50-99cf-ce4354a6bb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:38.128954 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db26ff5-6ec5-4c50-99cf-ce4354a6bb3f None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:38.129228 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db26ff5-6ec5-4c50-99cf-ce4354a6bb3f None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:38.129492 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db26ff5-6ec5-4c50-99cf-ce4354a6bb3f None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:38.130106 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db26ff5-6ec5-4c50-99cf-ce4354a6bb3f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:00:38.130400 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db26ff5-6ec5-4c50-99cf-ce4354a6bb3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:38.131773 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db26ff5-6ec5-4c50-99cf-ce4354a6bb3f None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:38.136601 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 485/976] 10.222.0.23 () {56 vars in 1116 bytes} [Thu Oct 13 13:00:37 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:00:38.142503 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e32946-6efe-4f1c-8d45-8aeedf806323 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:38.238513 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e32946-6efe-4f1c-8d45-8aeedf806323 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:38.270211 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab299a-32c7-4ede-8dfc-b3b2c63a0705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:38.270794 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab299a-32c7-4ede-8dfc-b3b2c63a0705 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:38.271098 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab299a-32c7-4ede-8dfc-b3b2c63a0705 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:38.271374 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab299a-32c7-4ede-8dfc-b3b2c63a0705 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:38.272064 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab299a-32c7-4ede-8dfc-b3b2c63a0705 None admin] RBAC: Adding query filter params (interface=public, service_id=81d47d6c911a4462bc76b57c0bad31d0, region_id=RegionOne) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:00:38.272366 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab299a-32c7-4ede-8dfc-b3b2c63a0705 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:38.273700 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab299a-32c7-4ede-8dfc-b3b2c63a0705 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:38.278465 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 492/977] 10.222.0.23 () {56 vars in 1251 bytes} [Thu Oct 13 13:00:38 2022] GET /identity/v3/endpoints?service_id=81d47d6c911a4462bc76b57c0bad31d0&interface=public®ion_id=RegionOne => generated 194 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:00:38.283868 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db26ff5-6ec5-4c50-99cf-ce4354a6bb3f None admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:38.287500 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:38.378743 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db26ff5-6ec5-4c50-99cf-ce4354a6bb3f None admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:38.409717 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc051f4-40f9-42bd-8f3f-1f7a19d5a8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:38.410304 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc051f4-40f9-42bd-8f3f-1f7a19d5a8e3 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:38.410579 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc051f4-40f9-42bd-8f3f-1f7a19d5a8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:38.410846 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc051f4-40f9-42bd-8f3f-1f7a19d5a8e3 None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:38.411466 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc051f4-40f9-42bd-8f3f-1f7a19d5a8e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:38.412817 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc051f4-40f9-42bd-8f3f-1f7a19d5a8e3 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:38.419594 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 486/978] 10.222.0.23 () {56 vars in 1087 bytes} [Thu Oct 13 13:00:38 2022] GET /identity/v3/services => generated 2191 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:00:40.540584 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73d04c-102d-4370-8624-1b498ea544bd None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:40.541132 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73d04c-102d-4370-8624-1b498ea544bd None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:40.541132 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73d04c-102d-4370-8624-1b498ea544bd None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:40.542467 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 493/979] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:00:40.547690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f8af1-5fb4-4567-80b4-129f8b867a5a None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:40.548095 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f8af1-5fb4-4567-80b4-129f8b867a5a None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:40.548234 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f8af1-5fb4-4567-80b4-129f8b867a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:40.567097 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-0a0f8af1-5fb4-4567-80b4-129f8b867a5a None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:00:40.568260 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-0a0f8af1-5fb4-4567-80b4-129f8b867a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:00:40.636670 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:40.642843 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 487/980] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:00:40 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 13 13:00:40.909576 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73d04c-102d-4370-8624-1b498ea544bd None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:40.930887 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:41.010443 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73d04c-102d-4370-8624-1b498ea544bd None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:41.042289 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68f065-6aaa-4848-9785-79daba79ca34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:41.042965 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68f065-6aaa-4848-9785-79daba79ca34 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:41.043273 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68f065-6aaa-4848-9785-79daba79ca34 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:41.043571 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68f065-6aaa-4848-9785-79daba79ca34 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:41.044148 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc68f065-6aaa-4848-9785-79daba79ca34 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:41.045492 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc68f065-6aaa-4848-9785-79daba79ca34 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:41.053855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-fc68f065-6aaa-4848-9785-79daba79ca34 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 13 13:00:41.053855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 13:00:41.053855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 13 13:00:41.053855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 13 13:00:41.053855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 13 13:00:41.053855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 13 13:00:41.053855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 13 13:00:41.053855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 13:00:41.053855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 13 13:00:41.053855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 13:00:41.053855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 13 13:00:41.053855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 13:00:41.053855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 13 13:00:41.053855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 13 13:00:41.053855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 13 13:00:41.053855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 13 13:00:41.053855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 13 13:00:41.055027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 13 13:00:41.055027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 13 13:00:41.055027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 13 13:00:41.055027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 13 13:00:41.055027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 13 13:00:41.055027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 13 13:00:41.055027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 13 13:00:41.055027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 13 13:00:41.055027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 13 13:00:41.055027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 13 13:00:41.055027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 13 13:00:41.055027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 13 13:00:41.055027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 13 13:00:41.055027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 13 13:00:41.055027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 13 13:00:41.055027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Oct 13 13:00:41.055903 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 13 13:00:41.055903 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Oct 13 13:00:41.055903 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 13 13:00:41.055903 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Oct 13 13:00:41.055903 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 13 13:00:41.055903 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 13 13:00:41.055903 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 13 13:00:41.055903 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Oct 13 13:00:41.055903 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 13 13:00:41.055903 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Oct 13 13:00:41.055903 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 13 13:00:41.055903 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 13 13:00:41.055903 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 13 13:00:41.055903 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Oct 13 13:00:41.055903 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 494/981] 10.222.0.23 () {56 vars in 1117 bytes} [Thu Oct 13 13:00:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 13 13:00:41.061382 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f8af1-5fb4-4567-80b4-129f8b867a5a None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:41.156029 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f8af1-5fb4-4567-80b4-129f8b867a5a None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:41.188527 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aa54ef-36c6-4333-9c0f-53d1496dfdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:41.189058 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa54ef-36c6-4333-9c0f-53d1496dfdc5 None admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:41.189856 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa54ef-36c6-4333-9c0f-53d1496dfdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:41.190125 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa54ef-36c6-4333-9c0f-53d1496dfdc5 None admin] PATH_INFO: `/v3/services` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:41.190739 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aa54ef-36c6-4333-9c0f-53d1496dfdc5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:00:41.191035 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aa54ef-36c6-4333-9c0f-53d1496dfdc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:41.192418 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aa54ef-36c6-4333-9c0f-53d1496dfdc5 None admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:41.197312 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 488/982] 10.222.0.23 () {56 vars in 1116 bytes} [Thu Oct 13 13:00:41 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:00:41.202955 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68f065-6aaa-4848-9785-79daba79ca34 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:00:41.280452 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:41.297765 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68f065-6aaa-4848-9785-79daba79ca34 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:00:41.330288 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a49e4c-001a-4ebf-8a49-0a089a40ffd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:00:41.330859 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a49e4c-001a-4ebf-8a49-0a089a40ffd2 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:00:41.331189 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a49e4c-001a-4ebf-8a49-0a089a40ffd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:00:41.331465 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a49e4c-001a-4ebf-8a49-0a089a40ffd2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:00:41.332121 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a49e4c-001a-4ebf-8a49-0a089a40ffd2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:00:41.333467 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a49e4c-001a-4ebf-8a49-0a089a40ffd2 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:00:41.351724 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 495/983] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:00:41 2022] POST /identity/v3/endpoints => generated 331 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:00:51.194415 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=69 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:00:51.347908 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:01:44.825726 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccf26a-b288-4c75-8584-b2716cf99857 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:01:44.826189 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccf26a-b288-4c75-8584-b2716cf99857 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:01:44.826296 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ccf26a-b288-4c75-8584-b2716cf99857 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:01:44.833424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9725032-0007-4f9c-b201-a67594d247a1 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:01:44.833849 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9725032-0007-4f9c-b201-a67594d247a1 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:01:44.833976 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9725032-0007-4f9c-b201-a67594d247a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:01:44.838014 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 489/984] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:01:44 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:01:44.961560 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b9725032-0007-4f9c-b201-a67594d247a1 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:01:44.962870 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b9725032-0007-4f9c-b201-a67594d247a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:01:45.099798 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 496/985] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:01:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:01:45.435460 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e44f4e-65a1-461a-ac9f-de756302b3ab None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:01:45.436573 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e44f4e-65a1-461a-ac9f-de756302b3ab None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:01:45.437014 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e44f4e-65a1-461a-ac9f-de756302b3ab None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:01:45.438779 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 490/986] 10.222.0.23 () {54 vars in 964 bytes} [Thu Oct 13 13:01:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:01:45.449848 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f65400-fd81-4a68-9e80-84c005905a9a None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:01:45.450420 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f65400-fd81-4a68-9e80-84c005905a9a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:01:45.450872 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f65400-fd81-4a68-9e80-84c005905a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:01:45.560718 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-79f65400-fd81-4a68-9e80-84c005905a9a None None] MFA Rules not processed for user `846251a661c04d3f9f195c4ac09b4189`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:01:45.560718 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-79f65400-fd81-4a68-9e80-84c005905a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:01:45.583300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:01:45.657886 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 497/987] 10.222.0.23 () {58 vars in 1061 bytes} [Thu Oct 13 13:01:45 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:01:45.667227 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e44f4e-65a1-461a-ac9f-de756302b3ab None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:01:45.668949 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-95e44f4e-65a1-461a-ac9f-de756302b3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:01:45.821576 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e44f4e-65a1-461a-ac9f-de756302b3ab None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:01:45.909331 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9231ba-900e-4a57-a9d5-a60f2554af7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:01:45.909946 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9231ba-900e-4a57-a9d5-a60f2554af7b service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:01:45.910498 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9231ba-900e-4a57-a9d5-a60f2554af7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:01:45.910498 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9231ba-900e-4a57-a9d5-a60f2554af7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:01:45.912076 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-ae9231ba-900e-4a57-a9d5-a60f2554af7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:01:45.976355 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:01:46.038483 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9231ba-900e-4a57-a9d5-a60f2554af7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:01:46.040079 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9231ba-900e-4a57-a9d5-a60f2554af7b service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:01:46.061084 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 491/988] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:01:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:01:48.479604 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864b0b9-f9ea-4282-b627-df0a647c6ef9 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:01:48.480162 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864b0b9-f9ea-4282-b627-df0a647c6ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:01:48.480162 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864b0b9-f9ea-4282-b627-df0a647c6ef9 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:01:48.481471 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 498/989] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:01:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:01:48.487322 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175bb92-7599-47d4-8606-47dbc11edad3 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:01:48.487764 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175bb92-7599-47d4-8606-47dbc11edad3 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:01:48.487866 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175bb92-7599-47d4-8606-47dbc11edad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:01:48.560606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-7175bb92-7599-47d4-8606-47dbc11edad3 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:01:48.561826 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-7175bb92-7599-47d4-8606-47dbc11edad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:01:48.657290 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 492/990] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:01:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:01:48.937607 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1864b0b9-f9ea-4282-b627-df0a647c6ef9 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:01:48.939232 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1864b0b9-f9ea-4282-b627-df0a647c6ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:01:49.039695 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1864b0b9-f9ea-4282-b627-df0a647c6ef9 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:01:49.108968 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:01:49.111268 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5feb3-ab51-4ce5-9f66-ed45b5e96128 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:01:49.112078 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5feb3-ab51-4ce5-9f66-ed45b5e96128 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:01:49.112529 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5feb3-ab51-4ce5-9f66-ed45b5e96128 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:01:49.112977 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5feb3-ab51-4ce5-9f66-ed45b5e96128 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:01:49.114593 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6bf5feb3-ab51-4ce5-9f66-ed45b5e96128 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:01:49.220406 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf5feb3-ab51-4ce5-9f66-ed45b5e96128 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:01:49.221756 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf5feb3-ab51-4ce5-9f66-ed45b5e96128 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:01:49.239064 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 499/991] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:01:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:01:51.660154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd13e68-54e1-462d-841a-373c0c8efd48 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:01:51.660683 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd13e68-54e1-462d-841a-373c0c8efd48 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:01:51.660683 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd13e68-54e1-462d-841a-373c0c8efd48 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:01:51.662027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 493/992] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:01:51.667276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb27f37-f40e-490d-9570-340c1490482e None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:01:51.667714 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb27f37-f40e-490d-9570-340c1490482e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:01:51.667832 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb27f37-f40e-490d-9570-340c1490482e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:01:51.731604 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-afb27f37-f40e-490d-9570-340c1490482e None None] MFA Rules not processed for user `a8e5d97876fd4bdfa107157ae3ae925b`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:01:51.732815 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-afb27f37-f40e-490d-9570-340c1490482e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:01:51.782470 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:01:51.825017 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 500/993] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:01:51 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:01:52.104188 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd13e68-54e1-462d-841a-373c0c8efd48 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:01:52.105696 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-bbd13e68-54e1-462d-841a-373c0c8efd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:01:52.116162 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:01:52.207720 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd13e68-54e1-462d-841a-373c0c8efd48 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:01:52.277155 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183cadd-38d7-49ed-83d7-c4d51b68a054 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:01:52.277743 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183cadd-38d7-49ed-83d7-c4d51b68a054 service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:01:52.278019 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183cadd-38d7-49ed-83d7-c4d51b68a054 service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:01:52.278284 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183cadd-38d7-49ed-83d7-c4d51b68a054 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:01:52.279975 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-b183cadd-38d7-49ed-83d7-c4d51b68a054 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:01:52.340054 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:01:52.386559 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b183cadd-38d7-49ed-83d7-c4d51b68a054 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:01:52.387956 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b183cadd-38d7-49ed-83d7-c4d51b68a054 service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:01:52.405201 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 494/994] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:01:52 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:01:55.066557 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d07447-3051-4bbd-978c-2dc6c4173684 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:01:55.067135 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d07447-3051-4bbd-978c-2dc6c4173684 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:01:55.067135 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d07447-3051-4bbd-978c-2dc6c4173684 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:01:55.068470 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 501/995] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:01:55.073606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0978a-8d6c-487c-a32b-f8536717b729 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:01:55.074038 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0978a-8d6c-487c-a32b-f8536717b729 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:01:55.074131 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d0978a-8d6c-487c-a32b-f8536717b729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:01:55.140666 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-02d0978a-8d6c-487c-a32b-f8536717b729 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:01:55.141876 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-02d0978a-8d6c-487c-a32b-f8536717b729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:01:55.236248 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 495/996] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:01:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:01:55.519000 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d07447-3051-4bbd-978c-2dc6c4173684 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:01:55.520557 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-37d07447-3051-4bbd-978c-2dc6c4173684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:01:55.620971 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d07447-3051-4bbd-978c-2dc6c4173684 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:01:55.691977 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e98678-837c-4d47-a1c2-17248532a29c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:01:55.692567 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e98678-837c-4d47-a1c2-17248532a29c service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:01:55.692845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e98678-837c-4d47-a1c2-17248532a29c service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:01:55.693117 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e98678-837c-4d47-a1c2-17248532a29c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:01:55.694666 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-81e98678-837c-4d47-a1c2-17248532a29c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:01:55.701254 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:01:55.800461 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e98678-837c-4d47-a1c2-17248532a29c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:01:55.801878 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e98678-837c-4d47-a1c2-17248532a29c service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:01:55.819915 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 502/997] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:01:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:01:58.595320 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d707c-3c6a-4a03-8f6d-028947613d90 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:01:58.595905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d707c-3c6a-4a03-8f6d-028947613d90 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:01:58.595905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d707c-3c6a-4a03-8f6d-028947613d90 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:01:58.597213 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 496/998] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:01:58.602440 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0281efe-9543-406f-92f8-da6405d39a36 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:01:58.602822 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0281efe-9543-406f-92f8-da6405d39a36 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:01:58.603205 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0281efe-9543-406f-92f8-da6405d39a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:01:58.672011 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d0281efe-9543-406f-92f8-da6405d39a36 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:01:58.673403 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d0281efe-9543-406f-92f8-da6405d39a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:01:58.735729 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:01:58.777072 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 503/999] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:01:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:01:59.060452 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d707c-3c6a-4a03-8f6d-028947613d90 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:01:59.061989 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-018d707c-3c6a-4a03-8f6d-028947613d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:01:59.082011 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:01:59.164882 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d707c-3c6a-4a03-8f6d-028947613d90 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:01:59.237945 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83784d4c-e884-4a3a-a7ab-e1de1714c90e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:01:59.238549 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83784d4c-e884-4a3a-a7ab-e1de1714c90e service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:01:59.238826 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83784d4c-e884-4a3a-a7ab-e1de1714c90e service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:01:59.239122 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83784d4c-e884-4a3a-a7ab-e1de1714c90e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:01:59.240700 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-83784d4c-e884-4a3a-a7ab-e1de1714c90e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:01:59.312781 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:01:59.351894 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83784d4c-e884-4a3a-a7ab-e1de1714c90e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:01:59.353246 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83784d4c-e884-4a3a-a7ab-e1de1714c90e service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:01:59.371219 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 497/1000] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:01:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:02:01.565371 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f41be2-8021-4616-a5ea-dd74016d99e9 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:01.565929 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f41be2-8021-4616-a5ea-dd74016d99e9 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:01.565929 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f41be2-8021-4616-a5ea-dd74016d99e9 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:01.567291 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 504/1001] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:02:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:02:01.572519 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbf236-67d7-446d-b6e5-a6a3c53ae3bf None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:01.572943 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbf236-67d7-446d-b6e5-a6a3c53ae3bf None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:01.573057 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbf236-67d7-446d-b6e5-a6a3c53ae3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:01.640597 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-24cbf236-67d7-446d-b6e5-a6a3c53ae3bf None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:02:01.641791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-24cbf236-67d7-446d-b6e5-a6a3c53ae3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:01.746251 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 498/1002] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:02:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:02:02.041983 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56d27b-5980-48cc-8786-d8628d9b81b5 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:02.042371 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56d27b-5980-48cc-8786-d8628d9b81b5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:02.042707 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56d27b-5980-48cc-8786-d8628d9b81b5 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:02.044162 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 505/1003] 10.222.0.23 () {54 vars in 964 bytes} [Thu Oct 13 13:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:02:02.050296 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d06e95-8c14-47c6-ab31-ab84bed2532f None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:02.050584 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d06e95-8c14-47c6-ab31-ab84bed2532f None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:02.050862 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d06e95-8c14-47c6-ab31-ab84bed2532f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:02.083156 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:02.127697 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-82d06e95-8c14-47c6-ab31-ab84bed2532f None None] MFA Rules not processed for user `846251a661c04d3f9f195c4ac09b4189`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:02:02.128902 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-82d06e95-8c14-47c6-ab31-ab84bed2532f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:02.204800 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 499/1004] 10.222.0.23 () {58 vars in 1061 bytes} [Thu Oct 13 13:02:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:02:02.213599 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e56d27b-5980-48cc-8786-d8628d9b81b5 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:02:02.215119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8e56d27b-5980-48cc-8786-d8628d9b81b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:02.318490 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e56d27b-5980-48cc-8786-d8628d9b81b5 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:02:02.390913 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0771f47-f86e-48f1-b3de-9754aa070644 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:02:02.391538 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0771f47-f86e-48f1-b3de-9754aa070644 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:02.391825 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0771f47-f86e-48f1-b3de-9754aa070644 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:02.392096 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0771f47-f86e-48f1-b3de-9754aa070644 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:02.393648 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a0771f47-f86e-48f1-b3de-9754aa070644 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:02.410200 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:02.502311 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0771f47-f86e-48f1-b3de-9754aa070644 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:02:02.503765 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0771f47-f86e-48f1-b3de-9754aa070644 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:02:02.521752 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 506/1005] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:02:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:02:04.974717 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd09dc-5df7-4b50-8def-bac551981acb None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:04.975250 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd09dc-5df7-4b50-8def-bac551981acb None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:04.975363 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd09dc-5df7-4b50-8def-bac551981acb None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:04.976640 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 500/1006] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:02:04.982162 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f3f30-c1b4-493a-a743-854c8c4988ba None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:04.982614 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f3f30-c1b4-493a-a743-854c8c4988ba None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:04.982705 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f3f30-c1b4-493a-a743-854c8c4988ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:05.052414 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-076f3f30-c1b4-493a-a743-854c8c4988ba None None] MFA Rules not processed for user `a8e5d97876fd4bdfa107157ae3ae925b`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:02:05.053629 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-076f3f30-c1b4-493a-a743-854c8c4988ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:05.120359 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:05.154459 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 507/1007] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:02:04 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:02:05.436930 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd09dc-5df7-4b50-8def-bac551981acb None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:02:05.438430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-06dd09dc-5df7-4b50-8def-bac551981acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:05.540544 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd09dc-5df7-4b50-8def-bac551981acb None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:02:05.560724 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:05.618425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de49aaef-cb17-4350-ad82-458d6174245f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:02:05.619031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49aaef-cb17-4350-ad82-458d6174245f service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:05.619366 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49aaef-cb17-4350-ad82-458d6174245f service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:05.619632 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49aaef-cb17-4350-ad82-458d6174245f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:05.621201 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-de49aaef-cb17-4350-ad82-458d6174245f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:05.733185 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de49aaef-cb17-4350-ad82-458d6174245f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:02:05.734569 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de49aaef-cb17-4350-ad82-458d6174245f service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:02:05.752880 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 501/1008] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:02:05 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:02:08.110304 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fb97f-94f2-4804-aa48-5169ae17e9ec None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:08.110908 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fb97f-94f2-4804-aa48-5169ae17e9ec None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:08.110908 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fb97f-94f2-4804-aa48-5169ae17e9ec None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:08.112216 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 508/1009] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:02:08.117445 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ebe90-d73d-4066-b9c8-d9c25dc7a56d None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:08.117835 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ebe90-d73d-4066-b9c8-d9c25dc7a56d None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:08.117991 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ebe90-d73d-4066-b9c8-d9c25dc7a56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:08.179523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:08.182646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-834ebe90-d73d-4066-b9c8-d9c25dc7a56d None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:02:08.184462 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-834ebe90-d73d-4066-b9c8-d9c25dc7a56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:08.277763 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 502/1010] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:02:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:02:08.558928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67fb97f-94f2-4804-aa48-5169ae17e9ec None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:02:08.560462 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c67fb97f-94f2-4804-aa48-5169ae17e9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:08.659424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67fb97f-94f2-4804-aa48-5169ae17e9ec None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:02:08.729590 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ffd02-a49f-409b-81d7-187466bdc639 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:02:08.730182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ffd02-a49f-409b-81d7-187466bdc639 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:08.730463 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ffd02-a49f-409b-81d7-187466bdc639 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:08.730732 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ffd02-a49f-409b-81d7-187466bdc639 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:08.732898 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3c2ffd02-a49f-409b-81d7-187466bdc639 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:08.758751 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:08.838256 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2ffd02-a49f-409b-81d7-187466bdc639 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:02:08.839703 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2ffd02-a49f-409b-81d7-187466bdc639 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:02:08.857819 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 509/1011] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:02:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:02:11.022497 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a0023-a09b-46c5-bb18-65fa62f06ac3 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:11.023077 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a0023-a09b-46c5-bb18-65fa62f06ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:11.023565 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a0023-a09b-46c5-bb18-65fa62f06ac3 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:11.026077 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 503/1012] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:02:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:02:11.031223 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ab602-8f1c-4f41-95b5-1635b759801d None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:11.031643 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ab602-8f1c-4f41-95b5-1635b759801d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:11.031748 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ab602-8f1c-4f41-95b5-1635b759801d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:11.095685 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-2c3ab602-8f1c-4f41-95b5-1635b759801d None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:02:11.096888 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2c3ab602-8f1c-4f41-95b5-1635b759801d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:11.167466 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:11.190028 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 510/1013] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:02:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:02:11.474323 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a0023-a09b-46c5-bb18-65fa62f06ac3 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:02:11.475919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-258a0023-a09b-46c5-bb18-65fa62f06ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:11.576564 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a0023-a09b-46c5-bb18-65fa62f06ac3 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:02:11.604569 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:11.647893 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2608831f-ac1c-49c9-b644-85f4496df789 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:02:11.648482 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608831f-ac1c-49c9-b644-85f4496df789 service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:11.648756 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608831f-ac1c-49c9-b644-85f4496df789 service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:11.649022 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608831f-ac1c-49c9-b644-85f4496df789 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:11.650593 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-2608831f-ac1c-49c9-b644-85f4496df789 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:11.754958 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2608831f-ac1c-49c9-b644-85f4496df789 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:02:11.756334 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2608831f-ac1c-49c9-b644-85f4496df789 service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:02:11.773867 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 504/1014] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:02:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:02:14.238550 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ae618-b52e-4886-b8f1-a982332d7de0 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:14.239155 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ae618-b52e-4886-b8f1-a982332d7de0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:14.239155 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ae618-b52e-4886-b8f1-a982332d7de0 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:14.240478 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 511/1015] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:02:14.245655 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d5f4d-3416-459b-abe1-7d6e92c207e1 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:14.246072 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d5f4d-3416-459b-abe1-7d6e92c207e1 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:14.246203 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d5f4d-3416-459b-abe1-7d6e92c207e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:14.310733 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-a93d5f4d-3416-459b-abe1-7d6e92c207e1 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:02:14.311945 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-a93d5f4d-3416-459b-abe1-7d6e92c207e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:14.319945 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:14.407172 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 505/1016] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:02:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:02:14.691757 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ae618-b52e-4886-b8f1-a982332d7de0 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:02:14.693258 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-470ae618-b52e-4886-b8f1-a982332d7de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:14.793050 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ae618-b52e-4886-b8f1-a982332d7de0 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:02:14.864012 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3eda39-0da9-4bcd-9d90-a821c20073d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:02:14.864602 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3eda39-0da9-4bcd-9d90-a821c20073d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:14.864878 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3eda39-0da9-4bcd-9d90-a821c20073d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:14.865143 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3eda39-0da9-4bcd-9d90-a821c20073d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:14.866674 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6d3eda39-0da9-4bcd-9d90-a821c20073d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:14.900505 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:14.978452 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3eda39-0da9-4bcd-9d90-a821c20073d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:02:14.979919 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3eda39-0da9-4bcd-9d90-a821c20073d8 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:02:14.997596 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 512/1017] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:02:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:02:19.023440 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241fe8fd-c693-4d91-8e8e-4b58b1b14be3 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:19.023982 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241fe8fd-c693-4d91-8e8e-4b58b1b14be3 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:19.024092 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241fe8fd-c693-4d91-8e8e-4b58b1b14be3 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:19.025339 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 506/1018] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:02:19.030663 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f80a5-46ae-48bc-826c-3288e9f6705f None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:19.031042 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f80a5-46ae-48bc-826c-3288e9f6705f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:19.031231 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f80a5-46ae-48bc-826c-3288e9f6705f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:19.098008 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-583f80a5-46ae-48bc-826c-3288e9f6705f None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:02:19.099166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-583f80a5-46ae-48bc-826c-3288e9f6705f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:19.183335 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:19.195098 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 513/1019] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:02:19.480795 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241fe8fd-c693-4d91-8e8e-4b58b1b14be3 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:02:19.482963 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-241fe8fd-c693-4d91-8e8e-4b58b1b14be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:19.587661 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241fe8fd-c693-4d91-8e8e-4b58b1b14be3 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:02:19.627227 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:19.663347 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93baa0-95c4-4052-855b-28cacc009775 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:02:19.663952 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93baa0-95c4-4052-855b-28cacc009775 service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:19.664231 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93baa0-95c4-4052-855b-28cacc009775 service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:19.664499 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93baa0-95c4-4052-855b-28cacc009775 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:19.666074 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-8b93baa0-95c4-4052-855b-28cacc009775 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:19.773020 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b93baa0-95c4-4052-855b-28cacc009775 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:02:19.774389 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b93baa0-95c4-4052-855b-28cacc009775 service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:02:19.792751 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 507/1020] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:02:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:02:22.582964 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b833c-a1f1-4b33-83e2-bc0ab57abb6b None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:22.583625 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b833c-a1f1-4b33-83e2-bc0ab57abb6b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:22.583625 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b833c-a1f1-4b33-83e2-bc0ab57abb6b None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:22.585074 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 514/1021] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:02:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:02:22.590420 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12f1c0-e5ff-43b0-bffd-041ae33f1a65 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:22.591399 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12f1c0-e5ff-43b0-bffd-041ae33f1a65 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:22.591492 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12f1c0-e5ff-43b0-bffd-041ae33f1a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:22.659161 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-8a12f1c0-e5ff-43b0-bffd-041ae33f1a65 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:02:22.660365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-8a12f1c0-e5ff-43b0-bffd-041ae33f1a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:22.678243 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:22.759475 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 508/1022] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:02:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:02:23.039162 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b833c-a1f1-4b33-83e2-bc0ab57abb6b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:02:23.040679 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-da1b833c-a1f1-4b33-83e2-bc0ab57abb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:23.149931 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b833c-a1f1-4b33-83e2-bc0ab57abb6b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:02:23.221298 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b7b9c-0dd2-4635-a054-8ab33c4f54fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:02:23.221814 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b7b9c-0dd2-4635-a054-8ab33c4f54fc service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:23.222089 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b7b9c-0dd2-4635-a054-8ab33c4f54fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:23.222355 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b7b9c-0dd2-4635-a054-8ab33c4f54fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:23.223946 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f92b7b9c-0dd2-4635-a054-8ab33c4f54fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:23.265532 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:23.333159 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92b7b9c-0dd2-4635-a054-8ab33c4f54fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:02:23.334602 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92b7b9c-0dd2-4635-a054-8ab33c4f54fc service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:02:23.353108 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 515/1023] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:02:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:02:27.471623 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53adea6f-e180-4ab1-9e3e-284a25115685 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:27.472148 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53adea6f-e180-4ab1-9e3e-284a25115685 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:27.472148 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53adea6f-e180-4ab1-9e3e-284a25115685 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:27.474111 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 509/1024] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:02:27.479121 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fd54d-c117-440f-a1a2-33be35ccc050 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:27.479529 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fd54d-c117-440f-a1a2-33be35ccc050 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:27.479735 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fd54d-c117-440f-a1a2-33be35ccc050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:27.548259 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8e0fd54d-c117-440f-a1a2-33be35ccc050 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:02:27.549641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8e0fd54d-c117-440f-a1a2-33be35ccc050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:27.644926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:27.647420 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 516/1025] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:02:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:02:27.930525 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53adea6f-e180-4ab1-9e3e-284a25115685 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:02:27.932350 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-53adea6f-e180-4ab1-9e3e-284a25115685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:28.036104 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53adea6f-e180-4ab1-9e3e-284a25115685 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:02:28.082980 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:28.107938 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b52c5-5e73-4710-b554-964330fa4bc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:02:28.108532 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b52c5-5e73-4710-b554-964330fa4bc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:28.108805 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b52c5-5e73-4710-b554-964330fa4bc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:28.109071 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b52c5-5e73-4710-b554-964330fa4bc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:28.110629 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-d87b52c5-5e73-4710-b554-964330fa4bc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:28.216574 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b52c5-5e73-4710-b554-964330fa4bc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:02:28.217914 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b52c5-5e73-4710-b554-964330fa4bc2 service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:02:28.235493 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 510/1026] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:02:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:02:33.315312 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59331bc-75b4-4492-8f01-495c1a2e0c39 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:33.315728 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59331bc-75b4-4492-8f01-495c1a2e0c39 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:33.316064 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59331bc-75b4-4492-8f01-495c1a2e0c39 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:33.317908 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 517/1027] 10.222.0.23 () {54 vars in 921 bytes} [Thu Oct 13 13:02:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:02:33.323461 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57f80e-5cc3-4a80-958b-a9fa3e5856d1 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:33.323895 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57f80e-5cc3-4a80-958b-a9fa3e5856d1 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:33.324107 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57f80e-5cc3-4a80-958b-a9fa3e5856d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:33.456187 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-6e57f80e-5cc3-4a80-958b-a9fa3e5856d1 None None] MFA Rules not processed for user `1a971cdb7f9d4e6692c6326c103d7303`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:02:33.457453 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-6e57f80e-5cc3-4a80-958b-a9fa3e5856d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:33.491233 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:33.593523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 511/1028] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Oct 13 13:02:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:02:37.545280 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f814f7-6567-4023-9fa3-6e5da77596af None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:37.546150 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f814f7-6567-4023-9fa3-6e5da77596af None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:37.547047 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f814f7-6567-4023-9fa3-6e5da77596af None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:37.548542 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 518/1029] 10.222.0.23 () {54 vars in 921 bytes} [Thu Oct 13 13:02:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:02:37.553544 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfb92a-aa62-48b0-b9b6-e7b1adee40cd None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:37.553963 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfb92a-aa62-48b0-b9b6-e7b1adee40cd None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:37.554223 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfb92a-aa62-48b0-b9b6-e7b1adee40cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:37.634559 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-a3bfb92a-aa62-48b0-b9b6-e7b1adee40cd None None] MFA Rules not processed for user `1a971cdb7f9d4e6692c6326c103d7303`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:02:37.635798 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-a3bfb92a-aa62-48b0-b9b6-e7b1adee40cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:37.709749 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:37.752272 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 512/1030] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Oct 13 13:02:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:02:39.256447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f43e1-4764-42b6-bd1d-87f9f16967b7 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:39.256996 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f43e1-4764-42b6-bd1d-87f9f16967b7 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:39.256996 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f43e1-4764-42b6-bd1d-87f9f16967b7 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:39.258238 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 519/1031] 10.222.0.23 () {54 vars in 921 bytes} [Thu Oct 13 13:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:02:39.264751 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bd9ec-e216-4f80-962d-b01c0a7c5e4c None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:39.265215 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bd9ec-e216-4f80-962d-b01c0a7c5e4c None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:39.265471 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bd9ec-e216-4f80-962d-b01c0a7c5e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:39.342419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-ad9bd9ec-e216-4f80-962d-b01c0a7c5e4c None None] MFA Rules not processed for user `1a971cdb7f9d4e6692c6326c103d7303`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:02:39.343648 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-ad9bd9ec-e216-4f80-962d-b01c0a7c5e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:39.449732 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 513/1032] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Oct 13 13:02:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:02:43.042097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4d50f-e7ff-43a3-9fb2-b63093c3b82d None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:43.042700 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4d50f-e7ff-43a3-9fb2-b63093c3b82d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:43.042700 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4d50f-e7ff-43a3-9fb2-b63093c3b82d None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:43.044106 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 520/1033] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:02:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:02:43.049491 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869028a5-35ca-4910-8c80-6dc306dc17a7 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:43.050260 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869028a5-35ca-4910-8c80-6dc306dc17a7 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:43.050545 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869028a5-35ca-4910-8c80-6dc306dc17a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:43.090324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:43.124771 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-869028a5-35ca-4910-8c80-6dc306dc17a7 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:02:43.126029 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-869028a5-35ca-4910-8c80-6dc306dc17a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:43.229356 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 514/1034] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:02:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:02:43.532371 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c298f-9b21-4f5a-86d1-2649c6f3238d None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:43.532978 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c298f-9b21-4f5a-86d1-2649c6f3238d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:43.533455 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c298f-9b21-4f5a-86d1-2649c6f3238d None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:43.535122 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 521/1035] 10.222.0.23 () {54 vars in 961 bytes} [Thu Oct 13 13:02:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:02:43.545285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39cd28-a73e-450a-a70a-903f0f81228c None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:43.546634 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39cd28-a73e-450a-a70a-903f0f81228c None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:43.548909 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39cd28-a73e-450a-a70a-903f0f81228c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:43.627452 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-be39cd28-a73e-450a-a70a-903f0f81228c None None] MFA Rules not processed for user `12700d985a724595af630c81e63f86d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:02:43.628944 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-be39cd28-a73e-450a-a70a-903f0f81228c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:43.690298 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:43.762130 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 515/1036] 10.222.0.23 () {58 vars in 1058 bytes} [Thu Oct 13 13:02:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:02:43.770749 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c298f-9b21-4f5a-86d1-2649c6f3238d None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:02:43.772457 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c23c298f-9b21-4f5a-86d1-2649c6f3238d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:43.939538 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c298f-9b21-4f5a-86d1-2649c6f3238d None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:02:44.017182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:44.062776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ed4953-8d60-4b31-94df-a16fff6a6f00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:02:44.063472 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed4953-8d60-4b31-94df-a16fff6a6f00 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:44.063673 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed4953-8d60-4b31-94df-a16fff6a6f00 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:44.063941 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed4953-8d60-4b31-94df-a16fff6a6f00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:44.065531 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-09ed4953-8d60-4b31-94df-a16fff6a6f00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:44.169343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ed4953-8d60-4b31-94df-a16fff6a6f00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:02:44.172506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ed4953-8d60-4b31-94df-a16fff6a6f00 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:02:44.191985 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 522/1037] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:02:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:02:46.572248 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ca533-b4ad-4360-8a8c-c6384c02f635 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:46.572825 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ca533-b4ad-4360-8a8c-c6384c02f635 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:46.572825 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ca533-b4ad-4360-8a8c-c6384c02f635 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:46.580016 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 516/1038] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:02:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:02:46.585830 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df454c24-c838-4ea2-a807-9ca3489afed7 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:46.586304 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df454c24-c838-4ea2-a807-9ca3489afed7 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:46.587368 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df454c24-c838-4ea2-a807-9ca3489afed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:46.673630 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-df454c24-c838-4ea2-a807-9ca3489afed7 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:02:46.674849 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-df454c24-c838-4ea2-a807-9ca3489afed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:46.715838 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:46.782296 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 523/1039] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:02:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:02:47.075782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ca533-b4ad-4360-8a8c-c6384c02f635 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:02:47.077618 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-1e8ca533-b4ad-4360-8a8c-c6384c02f635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:47.252062 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ca533-b4ad-4360-8a8c-c6384c02f635 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:02:47.264164 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:47.401422 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6a74e-f8a3-460e-822b-904bb77dfc5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:02:47.402628 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6a74e-f8a3-460e-822b-904bb77dfc5a service nova] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:47.402904 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6a74e-f8a3-460e-822b-904bb77dfc5a service nova] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:47.403205 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6a74e-f8a3-460e-822b-904bb77dfc5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:47.404783 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-c2e6a74e-f8a3-460e-822b-904bb77dfc5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:47.497068 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6a74e-f8a3-460e-822b-904bb77dfc5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:02:47.498577 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6a74e-f8a3-460e-822b-904bb77dfc5a service nova] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:02:47.517063 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 517/1040] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:02:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:02:49.999689 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad860f-c972-42a3-bd9e-7672bf7cb1c0 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:50.000197 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad860f-c972-42a3-bd9e-7672bf7cb1c0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:50.000197 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad860f-c972-42a3-bd9e-7672bf7cb1c0 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:50.001528 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 524/1041] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:02:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:02:50.006910 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe1ff1-d35b-40ea-9c1f-22e4bff96842 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:50.007361 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe1ff1-d35b-40ea-9c1f-22e4bff96842 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:50.007500 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe1ff1-d35b-40ea-9c1f-22e4bff96842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:50.023211 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:50.078044 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-fafe1ff1-d35b-40ea-9c1f-22e4bff96842 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:02:50.079278 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-fafe1ff1-d35b-40ea-9c1f-22e4bff96842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:50.183118 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 518/1042] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:02:50.474987 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae230f4-e520-4780-ae69-2c1df51af3a2 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:50.475318 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae230f4-e520-4780-ae69-2c1df51af3a2 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:50.475592 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae230f4-e520-4780-ae69-2c1df51af3a2 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:50.477114 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 525/1043] 10.222.0.23 () {54 vars in 961 bytes} [Thu Oct 13 13:02:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:02:50.482659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d60f3-628b-4343-88e7-6b2f22dd1016 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:50.482947 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d60f3-628b-4343-88e7-6b2f22dd1016 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:50.483249 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4d60f3-628b-4343-88e7-6b2f22dd1016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:50.562408 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-5d4d60f3-628b-4343-88e7-6b2f22dd1016 None None] MFA Rules not processed for user `12700d985a724595af630c81e63f86d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:02:50.563579 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-5d4d60f3-628b-4343-88e7-6b2f22dd1016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:50.600350 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:50.686531 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 519/1044] 10.222.0.23 () {58 vars in 1058 bytes} [Thu Oct 13 13:02:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:02:50.694498 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae230f4-e520-4780-ae69-2c1df51af3a2 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:02:50.696200 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4ae230f4-e520-4780-ae69-2c1df51af3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:50.852587 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae230f4-e520-4780-ae69-2c1df51af3a2 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:02:50.875624 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:50.994256 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b01881-864b-4132-afb6-c52c05c606ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:02:50.994864 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b01881-864b-4132-afb6-c52c05c606ba service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:50.995235 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b01881-864b-4132-afb6-c52c05c606ba service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:50.996008 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b01881-864b-4132-afb6-c52c05c606ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:50.998200 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c5b01881-864b-4132-afb6-c52c05c606ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:51.107576 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b01881-864b-4132-afb6-c52c05c606ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:02:51.109265 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b01881-864b-4132-afb6-c52c05c606ba service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:02:51.130716 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 526/1045] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:02:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:02:51.243160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eca5b3-3646-4bb7-b497-6b4d047c181c None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:51.243649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eca5b3-3646-4bb7-b497-6b4d047c181c None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:51.243729 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eca5b3-3646-4bb7-b497-6b4d047c181c None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:51.246155 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 520/1046] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:02:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:02:51.250985 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386dfff-8aeb-40ae-8659-9c84019ca991 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:51.251319 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386dfff-8aeb-40ae-8659-9c84019ca991 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:51.251576 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386dfff-8aeb-40ae-8659-9c84019ca991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:51.288954 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:51.332282 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-1386dfff-8aeb-40ae-8659-9c84019ca991 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:02:51.333768 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1386dfff-8aeb-40ae-8659-9c84019ca991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:51.447841 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 527/1047] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:02:51.779140 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86cfa5-e461-46be-9777-617fbdec5f72 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:51.779140 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86cfa5-e461-46be-9777-617fbdec5f72 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:51.779140 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86cfa5-e461-46be-9777-617fbdec5f72 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:51.780793 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 521/1048] 10.222.0.23 () {54 vars in 963 bytes} [Thu Oct 13 13:02:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:02:51.785879 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958d6b0-2268-4f54-a217-0cd449b6f4a9 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:51.786242 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958d6b0-2268-4f54-a217-0cd449b6f4a9 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:51.786589 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958d6b0-2268-4f54-a217-0cd449b6f4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:51.870508 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b958d6b0-2268-4f54-a217-0cd449b6f4a9 None None] MFA Rules not processed for user `2398a599165a46adb5718d98c897f238`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:02:51.872231 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b958d6b0-2268-4f54-a217-0cd449b6f4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:51.930273 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:51.962922 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 528/1049] 10.222.0.23 () {58 vars in 1060 bytes} [Thu Oct 13 13:02:51 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:02:51.971636 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be86cfa5-e461-46be-9777-617fbdec5f72 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:02:51.973522 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-be86cfa5-e461-46be-9777-617fbdec5f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:52.083475 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be86cfa5-e461-46be-9777-617fbdec5f72 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:02:52.133362 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:52.171955 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06c009-00a6-4dc8-97cc-1e95633e89e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:02:52.172717 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06c009-00a6-4dc8-97cc-1e95633e89e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:52.173104 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06c009-00a6-4dc8-97cc-1e95633e89e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:52.173496 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06c009-00a6-4dc8-97cc-1e95633e89e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:52.175363 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-ed06c009-00a6-4dc8-97cc-1e95633e89e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:52.309548 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed06c009-00a6-4dc8-97cc-1e95633e89e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:02:52.311282 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed06c009-00a6-4dc8-97cc-1e95633e89e6 service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:02:52.330416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 522/1050] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:02:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:02:53.396561 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182be43d-eeda-48b4-ad21-449b802eadf5 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:53.397116 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182be43d-eeda-48b4-ad21-449b802eadf5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:53.397116 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182be43d-eeda-48b4-ad21-449b802eadf5 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:53.398441 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 529/1051] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:02:53.403711 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a635c581-ee66-44f5-8828-50747f6df70c None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:53.404152 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a635c581-ee66-44f5-8828-50747f6df70c None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:53.404339 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a635c581-ee66-44f5-8828-50747f6df70c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:53.471301 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-a635c581-ee66-44f5-8828-50747f6df70c None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:02:53.472512 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-a635c581-ee66-44f5-8828-50747f6df70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:53.493099 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:53.568609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 523/1052] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:02:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:02:53.857452 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182be43d-eeda-48b4-ad21-449b802eadf5 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:02:53.859023 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-182be43d-eeda-48b4-ad21-449b802eadf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:54.018531 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182be43d-eeda-48b4-ad21-449b802eadf5 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:02:54.071189 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:54.135567 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9124e3-cddd-4c6c-9fbf-3aefeef699c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:02:54.136198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9124e3-cddd-4c6c-9fbf-3aefeef699c6 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:54.136482 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9124e3-cddd-4c6c-9fbf-3aefeef699c6 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:54.136754 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9124e3-cddd-4c6c-9fbf-3aefeef699c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:54.138339 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3f9124e3-cddd-4c6c-9fbf-3aefeef699c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:54.231134 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9124e3-cddd-4c6c-9fbf-3aefeef699c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:02:54.232709 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9124e3-cddd-4c6c-9fbf-3aefeef699c6 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:02:54.250453 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 530/1053] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:02:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:02:56.482983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eeb5ba-e593-4250-b709-a8173b531d96 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:56.483589 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eeb5ba-e593-4250-b709-a8173b531d96 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:56.483589 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eeb5ba-e593-4250-b709-a8173b531d96 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:56.484879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 524/1054] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:02:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:02:56.490068 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96aa887-abcd-49e8-b358-4b5b5f0b104e None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:56.490466 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96aa887-abcd-49e8-b358-4b5b5f0b104e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:56.490614 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96aa887-abcd-49e8-b358-4b5b5f0b104e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:56.561749 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b96aa887-abcd-49e8-b358-4b5b5f0b104e None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:02:56.563241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b96aa887-abcd-49e8-b358-4b5b5f0b104e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:56.571463 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:56.668396 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 531/1055] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:02:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:02:56.957105 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eeb5ba-e593-4250-b709-a8173b531d96 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:02:56.958907 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-36eeb5ba-e593-4250-b709-a8173b531d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:57.122909 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eeb5ba-e593-4250-b709-a8173b531d96 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:02:57.134768 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:57.244295 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e571b1a3-c73a-4442-b65f-f49d948c86d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:02:57.244849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571b1a3-c73a-4442-b65f-f49d948c86d6 service nova] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:57.245141 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571b1a3-c73a-4442-b65f-f49d948c86d6 service nova] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:57.245411 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571b1a3-c73a-4442-b65f-f49d948c86d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:57.246979 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-e571b1a3-c73a-4442-b65f-f49d948c86d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:57.340113 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e571b1a3-c73a-4442-b65f-f49d948c86d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:02:57.341701 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e571b1a3-c73a-4442-b65f-f49d948c86d6 service nova] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:02:57.367796 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 525/1056] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:02:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:02:59.587656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48c040-0b1e-4819-9dd3-fd8c161534d4 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:59.588125 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48c040-0b1e-4819-9dd3-fd8c161534d4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:59.588299 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48c040-0b1e-4819-9dd3-fd8c161534d4 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:59.590942 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 532/1057] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:02:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:02:59.595954 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a581e4-a68d-458e-ac05-a17639e383fc None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:02:59.596394 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a581e4-a68d-458e-ac05-a17639e383fc None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:02:59.596565 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a581e4-a68d-458e-ac05-a17639e383fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:02:59.608419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:02:59.678025 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-b1a581e4-a68d-458e-ac05-a17639e383fc None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:02:59.680077 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-b1a581e4-a68d-458e-ac05-a17639e383fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:02:59.806604 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 526/1058] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:02:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:03:00.124270 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48c040-0b1e-4819-9dd3-fd8c161534d4 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:03:00.125755 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4f48c040-0b1e-4819-9dd3-fd8c161534d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:00.272563 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:00.284602 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48c040-0b1e-4819-9dd3-fd8c161534d4 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:03:00.404481 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd6621b-4d01-45c1-aa4e-04ea742fe858 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:03:00.405027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6621b-4d01-45c1-aa4e-04ea742fe858 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:00.405306 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6621b-4d01-45c1-aa4e-04ea742fe858 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:00.405575 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6621b-4d01-45c1-aa4e-04ea742fe858 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:00.407464 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fbd6621b-4d01-45c1-aa4e-04ea742fe858 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:00.504791 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd6621b-4d01-45c1-aa4e-04ea742fe858 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:03:00.506516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd6621b-4d01-45c1-aa4e-04ea742fe858 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:03:00.517827 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:00.527920 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 533/1059] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:03:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:03:02.738048 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea87d4-2949-47c7-9189-3178ddd65eaa None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:02.738595 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea87d4-2949-47c7-9189-3178ddd65eaa None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:02.738595 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea87d4-2949-47c7-9189-3178ddd65eaa None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:02.740026 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 527/1060] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:03:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:03:02.746371 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a73664-78f3-45dd-a6bd-d4c151871458 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:02.746781 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a73664-78f3-45dd-a6bd-d4c151871458 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:02.746919 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a73664-78f3-45dd-a6bd-d4c151871458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:02.811579 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-76a73664-78f3-45dd-a6bd-d4c151871458 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:03:02.812797 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-76a73664-78f3-45dd-a6bd-d4c151871458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:02.910507 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 534/1061] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:03:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:03:03.200222 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea87d4-2949-47c7-9189-3178ddd65eaa None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:03:03.201757 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-86ea87d4-2949-47c7-9189-3178ddd65eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:03.308370 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:03.372245 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea87d4-2949-47c7-9189-3178ddd65eaa None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:03:03.487959 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38640ac7-6be5-443b-abf7-6749cdea9a2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:03:03.488486 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38640ac7-6be5-443b-abf7-6749cdea9a2c service nova] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:03.488763 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38640ac7-6be5-443b-abf7-6749cdea9a2c service nova] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:03.489037 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38640ac7-6be5-443b-abf7-6749cdea9a2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:03.490627 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-38640ac7-6be5-443b-abf7-6749cdea9a2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:03.540769 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:03.581765 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38640ac7-6be5-443b-abf7-6749cdea9a2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:03:03.583398 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38640ac7-6be5-443b-abf7-6749cdea9a2c service nova] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:03:03.601450 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 528/1062] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:03:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:03:05.766265 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728dfc9-80d3-4f75-8705-a345add702a5 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:05.766809 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728dfc9-80d3-4f75-8705-a345add702a5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:05.766809 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728dfc9-80d3-4f75-8705-a345add702a5 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:05.768266 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 535/1063] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:03:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:03:05.773702 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb90991-68be-4386-aae2-2d1f66afac9c None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:05.774135 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb90991-68be-4386-aae2-2d1f66afac9c None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:05.774255 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb90991-68be-4386-aae2-2d1f66afac9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:05.842109 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-5fb90991-68be-4386-aae2-2d1f66afac9c None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:03:05.843359 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-5fb90991-68be-4386-aae2-2d1f66afac9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:05.950389 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 529/1064] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:03:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:03:06.242449 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f728dfc9-80d3-4f75-8705-a345add702a5 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:03:06.244772 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f728dfc9-80d3-4f75-8705-a345add702a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:06.335798 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:06.411857 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f728dfc9-80d3-4f75-8705-a345add702a5 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:03:06.558508 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d528283-c7a2-46d0-a078-b3a4f27da67f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:03:06.559444 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d528283-c7a2-46d0-a078-b3a4f27da67f service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:06.559831 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d528283-c7a2-46d0-a078-b3a4f27da67f service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:06.560205 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d528283-c7a2-46d0-a078-b3a4f27da67f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:06.562604 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3d528283-c7a2-46d0-a078-b3a4f27da67f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:06.613039 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:06.667852 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d528283-c7a2-46d0-a078-b3a4f27da67f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:03:06.669454 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d528283-c7a2-46d0-a078-b3a4f27da67f service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:03:06.687871 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 536/1065] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:03:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:03:09.170572 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f27376-83bf-4d6b-abb5-7ff402e333cb None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:09.171160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f27376-83bf-4d6b-abb5-7ff402e333cb None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:09.171160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f27376-83bf-4d6b-abb5-7ff402e333cb None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:09.172479 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 530/1066] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:03:09.178132 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade357fe-89a1-4d34-91be-bde6b0b1eab4 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:09.178648 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade357fe-89a1-4d34-91be-bde6b0b1eab4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:09.178817 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade357fe-89a1-4d34-91be-bde6b0b1eab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:09.252510 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-ade357fe-89a1-4d34-91be-bde6b0b1eab4 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:03:09.253730 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ade357fe-89a1-4d34-91be-bde6b0b1eab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:09.376221 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 537/1067] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:03:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:03:09.666236 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f27376-83bf-4d6b-abb5-7ff402e333cb None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:03:09.667867 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-25f27376-83bf-4d6b-abb5-7ff402e333cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:09.701419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:09.823964 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f27376-83bf-4d6b-abb5-7ff402e333cb None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:03:09.940627 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:09.953251 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f85de0-68ba-45e9-a2d4-fcceacdb725d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:03:09.953864 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f85de0-68ba-45e9-a2d4-fcceacdb725d service nova] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:09.954139 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f85de0-68ba-45e9-a2d4-fcceacdb725d service nova] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:09.954408 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f85de0-68ba-45e9-a2d4-fcceacdb725d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:09.956011 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-a4f85de0-68ba-45e9-a2d4-fcceacdb725d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:10.048634 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f85de0-68ba-45e9-a2d4-fcceacdb725d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:03:10.050214 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f85de0-68ba-45e9-a2d4-fcceacdb725d service nova] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:03:10.068036 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 531/1068] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:03:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:03:11.535641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069024f-c25f-412f-a6d0-391cb9e2d8a0 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:11.536166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069024f-c25f-412f-a6d0-391cb9e2d8a0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:11.536166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069024f-c25f-412f-a6d0-391cb9e2d8a0 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:11.537516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 538/1069] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:03:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 13 13:03:11.543449 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028ca34-6b0e-46b5-9030-50bd2230b469 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:11.543863 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028ca34-6b0e-46b5-9030-50bd2230b469 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:11.544113 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028ca34-6b0e-46b5-9030-50bd2230b469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:11.610459 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-4028ca34-6b0e-46b5-9030-50bd2230b469 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:03:11.611670 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-4028ca34-6b0e-46b5-9030-50bd2230b469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:11.669172 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:11.708449 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 532/1070] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:03:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:03:12.006152 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3e599-9765-4ba5-87f8-6532b80cb13c None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:12.006439 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3e599-9765-4ba5-87f8-6532b80cb13c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:12.006718 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3e599-9765-4ba5-87f8-6532b80cb13c None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:12.008189 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 539/1071] 10.222.0.23 () {54 vars in 962 bytes} [Thu Oct 13 13:03:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:03:12.013300 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3257ef9-3c4e-468b-9b1c-7eea976e63b7 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:12.013686 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3257ef9-3c4e-468b-9b1c-7eea976e63b7 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:12.014022 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3257ef9-3c4e-468b-9b1c-7eea976e63b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:12.090319 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-c3257ef9-3c4e-468b-9b1c-7eea976e63b7 None None] MFA Rules not processed for user `45ce234c5b4643b2ac6341e0928f3830`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:03:12.091575 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-c3257ef9-3c4e-468b-9b1c-7eea976e63b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:12.173312 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 533/1072] 10.222.0.23 () {58 vars in 1059 bytes} [Thu Oct 13 13:03:12 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:03:12.183315 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3e599-9765-4ba5-87f8-6532b80cb13c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:03:12.184856 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-24d3e599-9765-4ba5-87f8-6532b80cb13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:12.200205 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:12.252708 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526447e-a1e0-495a-8e22-d0b82de3c2e3 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:12.252920 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526447e-a1e0-495a-8e22-d0b82de3c2e3 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:12.253290 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526447e-a1e0-495a-8e22-d0b82de3c2e3 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:12.254661 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 534/1073] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:03:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 13 13:03:12.259787 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf562551-dc87-4ce2-b898-ddddd882d35a None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:12.260075 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf562551-dc87-4ce2-b898-ddddd882d35a None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:12.260375 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf562551-dc87-4ce2-b898-ddddd882d35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:12.290327 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3e599-9765-4ba5-87f8-6532b80cb13c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:03:12.327869 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-cf562551-dc87-4ce2-b898-ddddd882d35a None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:03:12.329079 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-cf562551-dc87-4ce2-b898-ddddd882d35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:12.337330 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:12.364456 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b906d-f800-4a22-affa-cae967fb67be service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ce234c5b4643b2ac6341e0928f3830', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:03:12.365060 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b906d-f800-4a22-affa-cae967fb67be service glance] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:12.365341 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b906d-f800-4a22-affa-cae967fb67be service glance] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:12.365621 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b906d-f800-4a22-affa-cae967fb67be service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:12.367222 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-457b906d-f800-4a22-affa-cae967fb67be service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:12.428146 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 535/1074] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:03:12.436377 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:12.477017 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b906d-f800-4a22-affa-cae967fb67be service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:03:12.478420 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b906d-f800-4a22-affa-cae967fb67be service glance] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:03:12.496536 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 540/1075] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:03:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:03:12.720025 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf562551-dc87-4ce2-b898-ddddd882d35a None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:03:12.721266 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-cf562551-dc87-4ce2-b898-ddddd882d35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:12.838781 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf562551-dc87-4ce2-b898-ddddd882d35a None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:03:12.870205 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:12.965150 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2498eaa-b790-41cf-8725-80cac52ef3bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:03:12.965754 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2498eaa-b790-41cf-8725-80cac52ef3bc service nova] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:12.966026 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2498eaa-b790-41cf-8725-80cac52ef3bc service nova] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:12.966298 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2498eaa-b790-41cf-8725-80cac52ef3bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:12.967893 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-c2498eaa-b790-41cf-8725-80cac52ef3bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:13.061284 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2498eaa-b790-41cf-8725-80cac52ef3bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:03:13.062926 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2498eaa-b790-41cf-8725-80cac52ef3bc service nova] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:03:13.084119 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 536/1076] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:03:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:03:13.245664 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d7b32-fc23-4dd8-a2d8-e1f4c9eef0ef None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:13.246209 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d7b32-fc23-4dd8-a2d8-e1f4c9eef0ef None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:13.246360 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d7b32-fc23-4dd8-a2d8-e1f4c9eef0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:13.326291 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-909d7b32-fc23-4dd8-a2d8-e1f4c9eef0ef None None] MFA Rules not processed for user `b6ae674f0efc445fa970fca64e13893a`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:03:13.327536 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-909d7b32-fc23-4dd8-a2d8-e1f4c9eef0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:13.412277 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 541/1077] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Oct 13 13:03:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:03:13.466419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c088f-ae80-4992-be2a-e3dd6c4e0f80 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:13.466782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c088f-ae80-4992-be2a-e3dd6c4e0f80 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:13.467177 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c088f-ae80-4992-be2a-e3dd6c4e0f80 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:13.468540 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 537/1078] 10.222.0.23 () {54 vars in 943 bytes} [Thu Oct 13 13:03:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:03:13.475000 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ab75d-3804-4047-b55e-37203d7e602e None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:13.475318 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ab75d-3804-4047-b55e-37203d7e602e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:13.475597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ab75d-3804-4047-b55e-37203d7e602e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:13.517606 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:13.553054 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-374ab75d-3804-4047-b55e-37203d7e602e None None] MFA Rules not processed for user `526000341fd146db8c661f1fd14d4801`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:03:13.554257 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-374ab75d-3804-4047-b55e-37203d7e602e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:13.664231 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 542/1079] 10.222.0.23 () {58 vars in 1040 bytes} [Thu Oct 13 13:03:13 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:03:13.674511 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c088f-ae80-4992-be2a-e3dd6c4e0f80 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:03:13.676115 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-7a5c088f-ae80-4992-be2a-e3dd6c4e0f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:13.714223 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:13.817950 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c088f-ae80-4992-be2a-e3dd6c4e0f80 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:03:13.927167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a994de54-fcf7-4d3a-98aa-3b9331256824 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526000341fd146db8c661f1fd14d4801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:03:13.927703 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994de54-fcf7-4d3a-98aa-3b9331256824 service swift] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:13.927980 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994de54-fcf7-4d3a-98aa-3b9331256824 service swift] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:13.928246 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994de54-fcf7-4d3a-98aa-3b9331256824 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:13.929868 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-a994de54-fcf7-4d3a-98aa-3b9331256824 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:13.950287 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:14.017981 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a994de54-fcf7-4d3a-98aa-3b9331256824 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:03:14.019609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a994de54-fcf7-4d3a-98aa-3b9331256824 service swift] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:03:14.028620 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 538/1080] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 13 13:03:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 13 13:03:15.295210 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ecae2-c7a7-4df3-9b45-b0e7eda02d97 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:15.295780 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ecae2-c7a7-4df3-9b45-b0e7eda02d97 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:15.295780 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ecae2-c7a7-4df3-9b45-b0e7eda02d97 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:15.297144 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 543/1081] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:03:15.305140 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6016a4f-62c9-4d30-a3c6-1f9faf82acd7 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:15.306027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6016a4f-62c9-4d30-a3c6-1f9faf82acd7 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:15.306487 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6016a4f-62c9-4d30-a3c6-1f9faf82acd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:15.378992 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-c6016a4f-62c9-4d30-a3c6-1f9faf82acd7 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:03:15.380215 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-c6016a4f-62c9-4d30-a3c6-1f9faf82acd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:15.485287 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 539/1082] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:03:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:03:15.780012 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ecae2-c7a7-4df3-9b45-b0e7eda02d97 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:03:15.780770 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-847ecae2-c7a7-4df3-9b45-b0e7eda02d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:15.878654 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:15.944148 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ecae2-c7a7-4df3-9b45-b0e7eda02d97 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:03:16.068639 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600509d-dfa2-4f43-a403-4079e1d9ee9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:03:16.069256 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600509d-dfa2-4f43-a403-4079e1d9ee9f service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:16.069531 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600509d-dfa2-4f43-a403-4079e1d9ee9f service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:16.069813 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600509d-dfa2-4f43-a403-4079e1d9ee9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:16.071463 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0600509d-dfa2-4f43-a403-4079e1d9ee9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:16.133602 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:16.174233 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0600509d-dfa2-4f43-a403-4079e1d9ee9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:03:16.175864 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0600509d-dfa2-4f43-a403-4079e1d9ee9f service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:03:16.195800 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 544/1083] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:03:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:03:18.413755 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc3c69-f421-4ba8-b9e7-6285b04edc03 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:18.414328 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc3c69-f421-4ba8-b9e7-6285b04edc03 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:18.414328 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc3c69-f421-4ba8-b9e7-6285b04edc03 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:18.415701 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 540/1084] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:03:18.421642 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f39c46-5b30-4d47-9e6a-fa114956f67d None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:18.422084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f39c46-5b30-4d47-9e6a-fa114956f67d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:18.422169 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f39c46-5b30-4d47-9e6a-fa114956f67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:18.493432 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-12f39c46-5b30-4d47-9e6a-fa114956f67d None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:03:18.494977 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-12f39c46-5b30-4d47-9e6a-fa114956f67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:18.607127 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 545/1085] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:03:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:03:18.896473 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc3c69-f421-4ba8-b9e7-6285b04edc03 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:03:18.898060 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-98cc3c69-f421-4ba8-b9e7-6285b04edc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:18.908511 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:19.067521 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc3c69-f421-4ba8-b9e7-6285b04edc03 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:03:19.161605 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:19.202632 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399d7fe-8811-414a-9416-16f9e7a91e70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:03:19.203278 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399d7fe-8811-414a-9416-16f9e7a91e70 service nova] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:19.203564 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399d7fe-8811-414a-9416-16f9e7a91e70 service nova] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:19.203969 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399d7fe-8811-414a-9416-16f9e7a91e70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:19.205565 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-f399d7fe-8811-414a-9416-16f9e7a91e70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:19.306143 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f399d7fe-8811-414a-9416-16f9e7a91e70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:03:19.308592 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f399d7fe-8811-414a-9416-16f9e7a91e70 service nova] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:03:19.331334 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 541/1086] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:03:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:03:28.598542 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=86 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:29.328983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=58 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:36.626778 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220487b-5bad-41e5-b716-e5a2e5a5a4e8 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:36.627228 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220487b-5bad-41e5-b716-e5a2e5a5a4e8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:36.627360 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220487b-5bad-41e5-b716-e5a2e5a5a4e8 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:36.628766 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 546/1087] 10.222.0.23 () {54 vars in 918 bytes} [Thu Oct 13 13:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:03:36.633878 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef309e-af46-455b-997a-e4087bd5422b None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:36.634897 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef309e-af46-455b-997a-e4087bd5422b None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:36.634897 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef309e-af46-455b-997a-e4087bd5422b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:36.709770 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-c7ef309e-af46-455b-997a-e4087bd5422b None None] MFA Rules not processed for user `1a971cdb7f9d4e6692c6326c103d7303`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:03:36.710987 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-c7ef309e-af46-455b-997a-e4087bd5422b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:36.815721 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 542/1088] 10.222.0.23 () {58 vars in 1015 bytes} [Thu Oct 13 13:03:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:03:36.830507 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cce087-e60d-4c50-b2b4-907bca71417d None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:36.830797 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cce087-e60d-4c50-b2b4-907bca71417d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:36.831091 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cce087-e60d-4c50-b2b4-907bca71417d None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:36.832426 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 547/1089] 10.222.0.23 () {54 vars in 918 bytes} [Thu Oct 13 13:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:03:36.837028 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f142086-74eb-4917-b048-e5fba54f32bf None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:36.837416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f142086-74eb-4917-b048-e5fba54f32bf None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:36.838176 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f142086-74eb-4917-b048-e5fba54f32bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:36.908939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:36.912266 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-7f142086-74eb-4917-b048-e5fba54f32bf None None] MFA Rules not processed for user `12700d985a724595af630c81e63f86d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:03:36.914101 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-7f142086-74eb-4917-b048-e5fba54f32bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:37.028065 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 543/1090] 10.222.0.23 () {58 vars in 1015 bytes} [Thu Oct 13 13:03:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:03:39.585784 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ea094-84c6-47f9-b3fa-4bb1b4c094bf None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:39.586396 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ea094-84c6-47f9-b3fa-4bb1b4c094bf None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:39.586396 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ea094-84c6-47f9-b3fa-4bb1b4c094bf None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:39.587772 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 548/1091] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 13 13:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:03:39.593197 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6b056-3970-4244-b267-cfba3f5efcde None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:39.593630 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6b056-3970-4244-b267-cfba3f5efcde None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:39.593849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6b056-3970-4244-b267-cfba3f5efcde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:39.660090 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-d3d6b056-3970-4244-b267-cfba3f5efcde None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:03:39.661281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-d3d6b056-3970-4244-b267-cfba3f5efcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:39.715599 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:39.757045 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 544/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 13 13:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:03:40.045430 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ea094-84c6-47f9-b3fa-4bb1b4c094bf None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:03:40.046982 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3b5ea094-84c6-47f9-b3fa-4bb1b4c094bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:40.188228 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ea094-84c6-47f9-b3fa-4bb1b4c094bf None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:03:40.257677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:40.296895 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847227c-16dd-4667-b2dc-98975b82e47e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:03:40.297494 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847227c-16dd-4667-b2dc-98975b82e47e service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:40.297774 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847227c-16dd-4667-b2dc-98975b82e47e service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:40.298046 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847227c-16dd-4667-b2dc-98975b82e47e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:40.299713 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c847227c-16dd-4667-b2dc-98975b82e47e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:40.385968 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c847227c-16dd-4667-b2dc-98975b82e47e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:03:40.387568 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c847227c-16dd-4667-b2dc-98975b82e47e service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:03:40.405204 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 549/1093] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:03:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:03:41.536418 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8310f-675f-41b1-a279-a848f012b9c2 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:41.536980 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8310f-675f-41b1-a279-a848f012b9c2 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:41.536980 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8310f-675f-41b1-a279-a848f012b9c2 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:41.538719 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 545/1094] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 13 13:03:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:03:41.544821 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360da45-ee05-4f38-8132-30393fd16c10 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:41.545276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360da45-ee05-4f38-8132-30393fd16c10 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:41.545385 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360da45-ee05-4f38-8132-30393fd16c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:41.620562 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0360da45-ee05-4f38-8132-30393fd16c10 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:03:41.621772 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0360da45-ee05-4f38-8132-30393fd16c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:41.649131 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:41.716552 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 550/1095] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 13 13:03:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:03:41.867663 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a8310f-675f-41b1-a279-a848f012b9c2 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:03:41.869725 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-45a8310f-675f-41b1-a279-a848f012b9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:42.013497 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a8310f-675f-41b1-a279-a848f012b9c2 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:03:42.052713 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:42.124798 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62327db2-adce-4de2-8fd3-a50b19e01a1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:03:42.125398 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62327db2-adce-4de2-8fd3-a50b19e01a1a service nova] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:42.125669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62327db2-adce-4de2-8fd3-a50b19e01a1a service nova] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:42.125930 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62327db2-adce-4de2-8fd3-a50b19e01a1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:42.127528 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-62327db2-adce-4de2-8fd3-a50b19e01a1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:42.214638 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62327db2-adce-4de2-8fd3-a50b19e01a1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:03:42.216246 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62327db2-adce-4de2-8fd3-a50b19e01a1a service nova] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:03:42.234153 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 546/1096] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:03:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:03:43.358537 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230b5d1-d204-41b4-be74-28f78198ade4 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:43.359021 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230b5d1-d204-41b4-be74-28f78198ade4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:43.359143 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230b5d1-d204-41b4-be74-28f78198ade4 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:43.360461 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 551/1097] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 13 13:03:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:03:43.366718 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b70eb-6c1a-4e10-be6e-ea97022ca838 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:43.367191 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b70eb-6c1a-4e10-be6e-ea97022ca838 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:43.367322 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b70eb-6c1a-4e10-be6e-ea97022ca838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:43.420028 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:43.442631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-f83b70eb-6c1a-4e10-be6e-ea97022ca838 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:03:43.443786 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-f83b70eb-6c1a-4e10-be6e-ea97022ca838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:43.536182 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 547/1098] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 13 13:03:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:03:43.687221 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0230b5d1-d204-41b4-be74-28f78198ade4 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:03:43.688821 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0230b5d1-d204-41b4-be74-28f78198ade4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:43.830229 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0230b5d1-d204-41b4-be74-28f78198ade4 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:03:43.842043 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:43.941306 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130f77d-41d9-4d90-827a-8f35a7793a3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:03:43.941906 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130f77d-41d9-4d90-827a-8f35a7793a3a service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:43.942179 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130f77d-41d9-4d90-827a-8f35a7793a3a service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:43.942447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130f77d-41d9-4d90-827a-8f35a7793a3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:43.944038 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7130f77d-41d9-4d90-827a-8f35a7793a3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:44.031018 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7130f77d-41d9-4d90-827a-8f35a7793a3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:03:44.032627 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7130f77d-41d9-4d90-827a-8f35a7793a3a service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:03:44.050706 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 552/1099] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:03:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:03:45.167092 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48066a-c414-425e-bc60-bbeb71c2858f None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:45.167599 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48066a-c414-425e-bc60-bbeb71c2858f None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:45.167599 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48066a-c414-425e-bc60-bbeb71c2858f None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:45.169012 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 548/1100] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 13 13:03:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:03:45.175479 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3934b0a-4e26-40b9-8dad-9bba3e953133 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:45.175894 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3934b0a-4e26-40b9-8dad-9bba3e953133 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:45.176038 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3934b0a-4e26-40b9-8dad-9bba3e953133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:45.195407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:45.251602 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f3934b0a-4e26-40b9-8dad-9bba3e953133 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:03:45.252828 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f3934b0a-4e26-40b9-8dad-9bba3e953133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:45.347913 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 553/1101] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 13 13:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:03:45.497610 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48066a-c414-425e-bc60-bbeb71c2858f None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:03:45.499106 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-0c48066a-c414-425e-bc60-bbeb71c2858f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:45.608674 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:45.650583 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48066a-c414-425e-bc60-bbeb71c2858f None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:03:45.760803 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa6cd55-661b-402a-a2f4-2c649a0b50a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:03:45.761314 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa6cd55-661b-402a-a2f4-2c649a0b50a6 service nova] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:45.761592 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa6cd55-661b-402a-a2f4-2c649a0b50a6 service nova] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:45.761862 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa6cd55-661b-402a-a2f4-2c649a0b50a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:45.763467 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-7fa6cd55-661b-402a-a2f4-2c649a0b50a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:45.837893 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:45.851408 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa6cd55-661b-402a-a2f4-2c649a0b50a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:03:45.852978 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa6cd55-661b-402a-a2f4-2c649a0b50a6 service nova] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:03:45.871368 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 549/1102] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:03:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:03:46.992888 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df135b-5f25-4364-9238-d9c8b8ac2670 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:46.993401 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df135b-5f25-4364-9238-d9c8b8ac2670 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:46.993501 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df135b-5f25-4364-9238-d9c8b8ac2670 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:46.994804 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 554/1103] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 13 13:03:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:03:47.001374 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb948c-917a-4e2e-ae32-a10e34c02474 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:47.001815 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb948c-917a-4e2e-ae32-a10e34c02474 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:47.001916 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb948c-917a-4e2e-ae32-a10e34c02474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:47.078748 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-6dcb948c-917a-4e2e-ae32-a10e34c02474 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:03:47.079960 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-6dcb948c-917a-4e2e-ae32-a10e34c02474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:47.174373 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 550/1104] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 13 13:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:03:47.324598 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df135b-5f25-4364-9238-d9c8b8ac2670 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:03:47.326249 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d0df135b-5f25-4364-9238-d9c8b8ac2670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:47.409109 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:47.470035 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df135b-5f25-4364-9238-d9c8b8ac2670 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:03:47.580429 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf1293-6291-4cfc-912c-74924d059c89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:03:47.581032 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf1293-6291-4cfc-912c-74924d059c89 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:47.581313 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf1293-6291-4cfc-912c-74924d059c89 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:47.581586 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf1293-6291-4cfc-912c-74924d059c89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:47.583192 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ecaf1293-6291-4cfc-912c-74924d059c89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:47.631784 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:47.675573 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf1293-6291-4cfc-912c-74924d059c89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:03:47.677167 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf1293-6291-4cfc-912c-74924d059c89 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:03:47.695406 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 555/1105] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:03:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:03:48.752704 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603bb41e-7a6a-4fe5-8a14-a47223fa0867 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:48.753263 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603bb41e-7a6a-4fe5-8a14-a47223fa0867 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:48.753263 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603bb41e-7a6a-4fe5-8a14-a47223fa0867 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:48.754625 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 551/1106] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 13 13:03:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:03:48.760902 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0da31-935d-4352-bc7a-85859212911e None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:48.761430 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0da31-935d-4352-bc7a-85859212911e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:48.761430 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0da31-935d-4352-bc7a-85859212911e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:48.835568 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8ec0da31-935d-4352-bc7a-85859212911e None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:03:48.836782 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8ec0da31-935d-4352-bc7a-85859212911e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:48.932918 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 556/1107] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 13 13:03:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:03:49.080836 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603bb41e-7a6a-4fe5-8a14-a47223fa0867 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:03:49.082323 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-603bb41e-7a6a-4fe5-8a14-a47223fa0867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:49.130233 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:49.227854 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603bb41e-7a6a-4fe5-8a14-a47223fa0867 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:03:49.338877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b2454-e21c-4eba-91ff-98d5691a75c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:03:49.339506 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b2454-e21c-4eba-91ff-98d5691a75c8 service nova] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:49.339789 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b2454-e21c-4eba-91ff-98d5691a75c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:49.340065 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b2454-e21c-4eba-91ff-98d5691a75c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:49.341651 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-2d1b2454-e21c-4eba-91ff-98d5691a75c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:49.348479 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:49.432224 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1b2454-e21c-4eba-91ff-98d5691a75c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:03:49.433733 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1b2454-e21c-4eba-91ff-98d5691a75c8 service nova] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:03:49.451317 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 552/1108] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:03:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:03:50.509042 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fab878-2eca-497e-8878-355c088dac4b None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:50.509588 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fab878-2eca-497e-8878-355c088dac4b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:50.509588 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fab878-2eca-497e-8878-355c088dac4b None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:50.510962 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 557/1109] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 13 13:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:03:50.517076 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735fc600-7b1a-4b9f-8ad0-4793744fff50 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:50.517494 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735fc600-7b1a-4b9f-8ad0-4793744fff50 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:50.517632 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735fc600-7b1a-4b9f-8ad0-4793744fff50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:50.596417 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-735fc600-7b1a-4b9f-8ad0-4793744fff50 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:03:50.597841 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-735fc600-7b1a-4b9f-8ad0-4793744fff50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:50.665968 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:50.695231 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 553/1110] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 13 13:03:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:03:50.843711 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fab878-2eca-497e-8878-355c088dac4b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:03:50.845192 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a8fab878-2eca-497e-8878-355c088dac4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:50.862464 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:50.988377 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fab878-2eca-497e-8878-355c088dac4b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:03:51.074563 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:51.103559 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c2efb6-8dff-4f17-8f42-5d7707710294 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:03:51.104093 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2efb6-8dff-4f17-8f42-5d7707710294 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:51.104370 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2efb6-8dff-4f17-8f42-5d7707710294 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:51.104638 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2efb6-8dff-4f17-8f42-5d7707710294 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:51.106226 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d1c2efb6-8dff-4f17-8f42-5d7707710294 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:51.192657 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c2efb6-8dff-4f17-8f42-5d7707710294 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:03:51.194226 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c2efb6-8dff-4f17-8f42-5d7707710294 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:03:51.211899 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 558/1111] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:03:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:03:52.262589 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f8f29-bc9e-46c7-abdc-43811ae5a4da None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:52.263146 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f8f29-bc9e-46c7-abdc-43811ae5a4da None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:52.263146 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f8f29-bc9e-46c7-abdc-43811ae5a4da None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:52.264460 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 554/1112] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 13 13:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:03:52.270541 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa35f83-0ef2-47e0-b165-cd0d537d1dff None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:52.270929 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa35f83-0ef2-47e0-b165-cd0d537d1dff None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:52.271128 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa35f83-0ef2-47e0-b165-cd0d537d1dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:52.345989 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-caa35f83-0ef2-47e0-b165-cd0d537d1dff None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:03:52.347241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-caa35f83-0ef2-47e0-b165-cd0d537d1dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:52.386985 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:52.439136 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 559/1113] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 13 13:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:03:52.594314 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f8f29-bc9e-46c7-abdc-43811ae5a4da None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:03:52.596118 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-cd8f8f29-bc9e-46c7-abdc-43811ae5a4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:52.740477 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f8f29-bc9e-46c7-abdc-43811ae5a4da None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:03:52.793694 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:52.851034 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d889d71-b6ee-4c9d-b7c6-2dff4253ad65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:03:52.851606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d889d71-b6ee-4c9d-b7c6-2dff4253ad65 service nova] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:52.851883 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d889d71-b6ee-4c9d-b7c6-2dff4253ad65 service nova] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:52.852148 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d889d71-b6ee-4c9d-b7c6-2dff4253ad65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:52.854293 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-7d889d71-b6ee-4c9d-b7c6-2dff4253ad65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:52.943705 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d889d71-b6ee-4c9d-b7c6-2dff4253ad65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:03:52.945297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d889d71-b6ee-4c9d-b7c6-2dff4253ad65 service nova] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:03:52.963406 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 555/1114] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:03:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:03:54.005962 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44427830-7636-4b17-94b8-7f18419218d8 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:54.006510 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44427830-7636-4b17-94b8-7f18419218d8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:54.006510 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44427830-7636-4b17-94b8-7f18419218d8 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:54.007878 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 560/1115] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 13 13:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:03:54.014005 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1f687-9e11-4332-8cf5-4637decc055d None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:54.014475 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1f687-9e11-4332-8cf5-4637decc055d None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:54.014583 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1f687-9e11-4332-8cf5-4637decc055d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:54.089176 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-3db1f687-9e11-4332-8cf5-4637decc055d None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:03:54.090376 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-3db1f687-9e11-4332-8cf5-4637decc055d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:54.092366 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:54.186126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 556/1116] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 13 13:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:03:54.333150 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44427830-7636-4b17-94b8-7f18419218d8 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:03:54.334648 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-44427830-7636-4b17-94b8-7f18419218d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:54.475222 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44427830-7636-4b17-94b8-7f18419218d8 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:03:54.502465 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:54.586880 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40ee00-ab20-4086-b8ed-df74fc95826e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:03:54.588060 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40ee00-ab20-4086-b8ed-df74fc95826e service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:54.588339 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40ee00-ab20-4086-b8ed-df74fc95826e service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:54.588612 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40ee00-ab20-4086-b8ed-df74fc95826e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:54.590203 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ca40ee00-ab20-4086-b8ed-df74fc95826e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:54.676588 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca40ee00-ab20-4086-b8ed-df74fc95826e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:03:54.678238 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca40ee00-ab20-4086-b8ed-df74fc95826e service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:03:54.695921 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 561/1117] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:03:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:03:56.723707 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459eef2-9728-4277-9b57-423d5bd96011 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:56.724287 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459eef2-9728-4277-9b57-423d5bd96011 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:56.724287 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459eef2-9728-4277-9b57-423d5bd96011 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:56.725577 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 557/1118] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 13 13:03:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:03:56.730908 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495988a-b5b3-44bb-afbd-f69349b6d032 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:56.731352 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495988a-b5b3-44bb-afbd-f69349b6d032 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:56.731522 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495988a-b5b3-44bb-afbd-f69349b6d032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:56.772031 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:56.798483 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0495988a-b5b3-44bb-afbd-f69349b6d032 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:03:56.800039 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0495988a-b5b3-44bb-afbd-f69349b6d032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:56.894166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 562/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 13 13:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:03:57.176075 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459eef2-9728-4277-9b57-423d5bd96011 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:03:57.177580 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-2459eef2-9728-4277-9b57-423d5bd96011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:57.278572 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459eef2-9728-4277-9b57-423d5bd96011 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:03:57.310054 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:57.350742 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95d3b6-ab77-42fe-a2d6-e19af34ea776 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:03:57.351360 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95d3b6-ab77-42fe-a2d6-e19af34ea776 service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:57.351638 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95d3b6-ab77-42fe-a2d6-e19af34ea776 service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:57.351907 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95d3b6-ab77-42fe-a2d6-e19af34ea776 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:57.353459 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-bc95d3b6-ab77-42fe-a2d6-e19af34ea776 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:57.459539 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc95d3b6-ab77-42fe-a2d6-e19af34ea776 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:03:57.460892 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc95d3b6-ab77-42fe-a2d6-e19af34ea776 service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:03:57.478598 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 558/1120] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:03:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:03:59.810437 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fca004-5b46-4b30-a2a8-0b760bdb8409 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:59.811023 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fca004-5b46-4b30-a2a8-0b760bdb8409 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:59.811023 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fca004-5b46-4b30-a2a8-0b760bdb8409 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:59.812423 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 563/1121] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 13 13:03:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:03:59.817629 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fda96-7a25-49ca-9e26-9ae545cfed1d None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:03:59.818060 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fda96-7a25-49ca-9e26-9ae545cfed1d None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:03:59.818164 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fda96-7a25-49ca-9e26-9ae545cfed1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:03:59.881752 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-225fda96-7a25-49ca-9e26-9ae545cfed1d None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:03:59.882931 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-225fda96-7a25-49ca-9e26-9ae545cfed1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:03:59.895792 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:03:59.978906 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 559/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 13 13:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:04:00.267931 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fca004-5b46-4b30-a2a8-0b760bdb8409 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:04:00.269456 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-24fca004-5b46-4b30-a2a8-0b760bdb8409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:00.369634 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fca004-5b46-4b30-a2a8-0b760bdb8409 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:04:00.382450 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:04:00.449527 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dcf285-c78e-4527-a5ea-a42fdcc695fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:04:00.450132 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dcf285-c78e-4527-a5ea-a42fdcc695fa service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:00.450414 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dcf285-c78e-4527-a5ea-a42fdcc695fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:00.450687 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dcf285-c78e-4527-a5ea-a42fdcc695fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:00.452311 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e7dcf285-c78e-4527-a5ea-a42fdcc695fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:00.559104 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dcf285-c78e-4527-a5ea-a42fdcc695fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:04:00.560531 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dcf285-c78e-4527-a5ea-a42fdcc695fa service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:04:00.577894 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 564/1123] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:04:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:04:03.086587 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45de20-1472-4415-b8a4-dd634b8e29c5 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:03.087181 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45de20-1472-4415-b8a4-dd634b8e29c5 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:03.087181 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45de20-1472-4415-b8a4-dd634b8e29c5 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:03.088523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 560/1124] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 13 13:04:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:04:03.093781 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23a849-064f-42da-8c7c-cfd5f01848c2 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:03.094179 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23a849-064f-42da-8c7c-cfd5f01848c2 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:03.094333 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23a849-064f-42da-8c7c-cfd5f01848c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:03.141291 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:04:03.160108 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-4c23a849-064f-42da-8c7c-cfd5f01848c2 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:04:03.161257 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4c23a849-064f-42da-8c7c-cfd5f01848c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:03.254527 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 565/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 13 13:04:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:04:03.534253 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45de20-1472-4415-b8a4-dd634b8e29c5 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:04:03.535832 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-1e45de20-1472-4415-b8a4-dd634b8e29c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:03.640303 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45de20-1472-4415-b8a4-dd634b8e29c5 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:04:03.680200 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:04:03.714297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee66c7-aaf4-44d7-b082-4c5bf78a1191 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:04:03.714890 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee66c7-aaf4-44d7-b082-4c5bf78a1191 service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:03.715201 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee66c7-aaf4-44d7-b082-4c5bf78a1191 service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:03.715471 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee66c7-aaf4-44d7-b082-4c5bf78a1191 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:03.717034 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-e9ee66c7-aaf4-44d7-b082-4c5bf78a1191 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:03.822762 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee66c7-aaf4-44d7-b082-4c5bf78a1191 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:04:03.824224 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee66c7-aaf4-44d7-b082-4c5bf78a1191 service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:04:03.842035 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 561/1126] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:04:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:04:06.076786 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df832d7-9c99-49f7-9b40-f62fe32b8038 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:06.077346 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df832d7-9c99-49f7-9b40-f62fe32b8038 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:06.077346 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df832d7-9c99-49f7-9b40-f62fe32b8038 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:06.078658 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 566/1127] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 13 13:04:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:04:06.084315 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebce705-dbb2-41bc-bd5e-6d5fa11b3fb6 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:06.084766 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebce705-dbb2-41bc-bd5e-6d5fa11b3fb6 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:06.084866 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebce705-dbb2-41bc-bd5e-6d5fa11b3fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:06.150150 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-cebce705-dbb2-41bc-bd5e-6d5fa11b3fb6 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:04:06.151401 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-cebce705-dbb2-41bc-bd5e-6d5fa11b3fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:06.171286 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:04:06.246826 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 562/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 13 13:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:04:06.530080 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df832d7-9c99-49f7-9b40-f62fe32b8038 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:04:06.531718 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7df832d7-9c99-49f7-9b40-f62fe32b8038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:06.632626 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df832d7-9c99-49f7-9b40-f62fe32b8038 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:04:06.654099 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:04:06.707757 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e0721-64cb-42ad-9425-d8e455539529 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:04:06.708357 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e0721-64cb-42ad-9425-d8e455539529 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:06.708642 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e0721-64cb-42ad-9425-d8e455539529 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:06.708914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e0721-64cb-42ad-9425-d8e455539529 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:06.710487 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6a0e0721-64cb-42ad-9425-d8e455539529 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:06.816143 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e0721-64cb-42ad-9425-d8e455539529 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:04:06.817564 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e0721-64cb-42ad-9425-d8e455539529 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:04:06.835429 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 567/1129] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:04:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:04:09.022642 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f69a1-248b-49e9-82ad-e13fa94db2f2 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:09.023251 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f69a1-248b-49e9-82ad-e13fa94db2f2 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:09.023251 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f69a1-248b-49e9-82ad-e13fa94db2f2 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:09.024597 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 563/1130] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 13 13:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:04:09.030043 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb98bd0-ffcf-4315-bc68-9c01627e3e79 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:09.030461 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb98bd0-ffcf-4315-bc68-9c01627e3e79 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:09.030593 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb98bd0-ffcf-4315-bc68-9c01627e3e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:09.093665 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:04:09.096578 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-4bb98bd0-ffcf-4315-bc68-9c01627e3e79 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:04:09.098377 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4bb98bd0-ffcf-4315-bc68-9c01627e3e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:09.193714 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 568/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 13 13:04:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:04:09.481063 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f69a1-248b-49e9-82ad-e13fa94db2f2 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:04:09.482582 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-5a8f69a1-248b-49e9-82ad-e13fa94db2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:09.582819 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f69a1-248b-49e9-82ad-e13fa94db2f2 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:04:09.635451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:04:09.660154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa225a45-48b6-4722-9c60-7a374b5ae409 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:04:09.660748 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa225a45-48b6-4722-9c60-7a374b5ae409 service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:09.661022 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa225a45-48b6-4722-9c60-7a374b5ae409 service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:09.661288 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa225a45-48b6-4722-9c60-7a374b5ae409 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:09.663171 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-fa225a45-48b6-4722-9c60-7a374b5ae409 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:09.770172 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa225a45-48b6-4722-9c60-7a374b5ae409 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:04:09.771632 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa225a45-48b6-4722-9c60-7a374b5ae409 service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:04:09.789001 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 564/1132] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:04:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:04:11.960908 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56573a2-3b2b-41c4-8ec1-2e685295e20a None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:11.961461 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56573a2-3b2b-41c4-8ec1-2e685295e20a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:11.961461 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56573a2-3b2b-41c4-8ec1-2e685295e20a None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:11.962814 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 569/1133] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 13 13:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:04:11.968258 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d923a5-b140-477c-b1f9-257b9ad68fe8 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:11.968675 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d923a5-b140-477c-b1f9-257b9ad68fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:11.968812 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d923a5-b140-477c-b1f9-257b9ad68fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:12.034782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-c6d923a5-b140-477c-b1f9-257b9ad68fe8 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:04:12.036219 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-c6d923a5-b140-477c-b1f9-257b9ad68fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:12.065211 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:04:12.130403 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 565/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 13 13:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:04:12.411153 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56573a2-3b2b-41c4-8ec1-2e685295e20a None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:04:12.412655 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f56573a2-3b2b-41c4-8ec1-2e685295e20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:12.513960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56573a2-3b2b-41c4-8ec1-2e685295e20a None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:04:12.541409 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:04:12.584880 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf1c5e0-e4ba-4c80-b382-9ceb4aba1e8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:04:12.585476 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1c5e0-e4ba-4c80-b382-9ceb4aba1e8f service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:12.585749 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1c5e0-e4ba-4c80-b382-9ceb4aba1e8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:12.586024 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1c5e0-e4ba-4c80-b382-9ceb4aba1e8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:12.587647 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-dbf1c5e0-e4ba-4c80-b382-9ceb4aba1e8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:12.696278 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf1c5e0-e4ba-4c80-b382-9ceb4aba1e8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:04:12.697624 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf1c5e0-e4ba-4c80-b382-9ceb4aba1e8f service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:04:12.714900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 570/1135] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:04:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:04:14.928926 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4721809-d284-4ac4-afb1-04552daef1c3 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:14.929456 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4721809-d284-4ac4-afb1-04552daef1c3 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:14.929456 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4721809-d284-4ac4-afb1-04552daef1c3 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:14.931168 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 566/1136] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 13 13:04:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:04:14.936958 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b1d7f-9500-4c98-9db1-f6ac37bce98f None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:14.937472 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b1d7f-9500-4c98-9db1-f6ac37bce98f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:14.937566 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b1d7f-9500-4c98-9db1-f6ac37bce98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:15.002033 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-491b1d7f-9500-4c98-9db1-f6ac37bce98f None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:04:15.003272 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-491b1d7f-9500-4c98-9db1-f6ac37bce98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:15.011926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:04:15.099607 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 571/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 13 13:04:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:04:15.380538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4721809-d284-4ac4-afb1-04552daef1c3 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:04:15.382083 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-e4721809-d284-4ac4-afb1-04552daef1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:15.483132 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4721809-d284-4ac4-afb1-04552daef1c3 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:04:15.537680 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:04:15.556800 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea3398-9911-467b-8c14-f582c1bc29df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:04:15.557417 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea3398-9911-467b-8c14-f582c1bc29df service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:15.557685 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea3398-9911-467b-8c14-f582c1bc29df service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:15.557955 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea3398-9911-467b-8c14-f582c1bc29df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:15.559545 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-e7ea3398-9911-467b-8c14-f582c1bc29df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:15.667025 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea3398-9911-467b-8c14-f582c1bc29df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:04:15.668414 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea3398-9911-467b-8c14-f582c1bc29df service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:04:15.686818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 567/1138] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:04:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:04:17.881404 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b070ed-c03e-48b0-b76c-b69b89f3f31a None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:17.882004 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b070ed-c03e-48b0-b76c-b69b89f3f31a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:17.882004 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b070ed-c03e-48b0-b76c-b69b89f3f31a None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:17.883333 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 572/1139] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 13 13:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:04:17.888672 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f25b2-8b3e-4dee-a23a-b503e8fbff9e None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:17.889090 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f25b2-8b3e-4dee-a23a-b503e8fbff9e None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:17.889217 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f25b2-8b3e-4dee-a23a-b503e8fbff9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:17.955715 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-815f25b2-8b3e-4dee-a23a-b503e8fbff9e None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:04:17.956915 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-815f25b2-8b3e-4dee-a23a-b503e8fbff9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:17.995044 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:04:18.052046 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 568/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 13 13:04:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:04:18.339495 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b070ed-c03e-48b0-b76c-b69b89f3f31a None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:04:18.341022 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c6b070ed-c03e-48b0-b76c-b69b89f3f31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:18.441749 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b070ed-c03e-48b0-b76c-b69b89f3f31a None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:04:18.480461 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:04:18.516746 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c87b555-3ca4-4d5b-95ad-849786a98b61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:04:18.517346 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87b555-3ca4-4d5b-95ad-849786a98b61 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:18.517622 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87b555-3ca4-4d5b-95ad-849786a98b61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:18.517890 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87b555-3ca4-4d5b-95ad-849786a98b61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:18.519470 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2c87b555-3ca4-4d5b-95ad-849786a98b61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:18.626744 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c87b555-3ca4-4d5b-95ad-849786a98b61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:04:18.628124 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c87b555-3ca4-4d5b-95ad-849786a98b61 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:04:18.646159 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 573/1141] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:04:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:04:20.800964 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efab64a-a4d2-458a-b505-7ce479b5961d None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:20.801516 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efab64a-a4d2-458a-b505-7ce479b5961d None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:20.801516 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efab64a-a4d2-458a-b505-7ce479b5961d None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:20.802837 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 569/1142] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 13 13:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:04:20.808651 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d475920-6245-476c-8eb8-bb1b357191af None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:20.809101 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d475920-6245-476c-8eb8-bb1b357191af None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:20.809199 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d475920-6245-476c-8eb8-bb1b357191af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:20.873191 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-4d475920-6245-476c-8eb8-bb1b357191af None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:04:20.874398 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4d475920-6245-476c-8eb8-bb1b357191af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:20.892734 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:04:20.971061 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 574/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 13 13:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:04:21.253109 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efab64a-a4d2-458a-b505-7ce479b5961d None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:04:21.254657 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-3efab64a-a4d2-458a-b505-7ce479b5961d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:21.356688 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efab64a-a4d2-458a-b505-7ce479b5961d None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:04:21.422122 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:04:21.429026 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd8616-2dc6-487c-90bd-28d450997fad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:04:21.429624 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd8616-2dc6-487c-90bd-28d450997fad service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:21.430451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd8616-2dc6-487c-90bd-28d450997fad service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:21.430719 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd8616-2dc6-487c-90bd-28d450997fad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:21.432607 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-fafd8616-2dc6-487c-90bd-28d450997fad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:21.538165 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd8616-2dc6-487c-90bd-28d450997fad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:04:21.540066 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd8616-2dc6-487c-90bd-28d450997fad service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:04:21.558278 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 570/1144] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:04:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:04:23.698326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a3499-ea86-4056-b562-b3bdad8cdedf None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:23.698742 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a3499-ea86-4056-b562-b3bdad8cdedf None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:23.698926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a3499-ea86-4056-b562-b3bdad8cdedf None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:23.700296 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 575/1145] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 13 13:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:04:23.705652 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f3007-62c9-4b7d-b2d2-61c89561812a None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:23.706069 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f3007-62c9-4b7d-b2d2-61c89561812a None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:23.706237 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f3007-62c9-4b7d-b2d2-61c89561812a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:23.770989 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-ad4f3007-62c9-4b7d-b2d2-61c89561812a None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:04:23.772193 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-ad4f3007-62c9-4b7d-b2d2-61c89561812a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:23.820849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:04:23.870156 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 571/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 13 13:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:04:24.150278 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a3499-ea86-4056-b562-b3bdad8cdedf None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:04:24.151884 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-366a3499-ea86-4056-b562-b3bdad8cdedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:24.251597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a3499-ea86-4056-b562-b3bdad8cdedf None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:04:24.298356 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:04:24.323745 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0cd511-bbfd-4be5-9da7-47dd95e57e09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:04:24.324351 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0cd511-bbfd-4be5-9da7-47dd95e57e09 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:24.324635 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0cd511-bbfd-4be5-9da7-47dd95e57e09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:24.324910 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0cd511-bbfd-4be5-9da7-47dd95e57e09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:24.326486 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8f0cd511-bbfd-4be5-9da7-47dd95e57e09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:24.433764 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0cd511-bbfd-4be5-9da7-47dd95e57e09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:04:24.435174 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0cd511-bbfd-4be5-9da7-47dd95e57e09 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:04:24.453372 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 576/1147] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:04:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:04:33.868598 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=22 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:04:34.451196 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=66 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:04:44.219394 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05945ae-3b41-4290-870e-d466ea6d940f None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:44.219954 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05945ae-3b41-4290-870e-d466ea6d940f None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:44.219954 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05945ae-3b41-4290-870e-d466ea6d940f None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:44.221294 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 572/1148] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:04:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:04:44.226514 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70e5d6-9d1d-4aed-a26c-8d7e8432e8f2 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:44.226943 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70e5d6-9d1d-4aed-a26c-8d7e8432e8f2 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:44.227084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70e5d6-9d1d-4aed-a26c-8d7e8432e8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:44.290398 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-7a70e5d6-9d1d-4aed-a26c-8d7e8432e8f2 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:04:44.291634 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7a70e5d6-9d1d-4aed-a26c-8d7e8432e8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:44.384159 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 577/1149] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:04:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:04:44.682131 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73a59f-7c6a-460f-b298-7bb943655502 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:44.682420 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73a59f-7c6a-460f-b298-7bb943655502 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:44.682715 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73a59f-7c6a-460f-b298-7bb943655502 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:44.684128 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 573/1150] 10.222.0.23 () {54 vars in 962 bytes} [Thu Oct 13 13:04:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:04:44.689167 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94ed22-da39-4de3-88ff-8c8e27e555f0 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:44.689459 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94ed22-da39-4de3-88ff-8c8e27e555f0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:44.689769 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94ed22-da39-4de3-88ff-8c8e27e555f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:44.763731 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-fd94ed22-da39-4de3-88ff-8c8e27e555f0 None None] MFA Rules not processed for user `45ce234c5b4643b2ac6341e0928f3830`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:04:44.764927 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fd94ed22-da39-4de3-88ff-8c8e27e555f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:44.784652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:04:44.842763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 578/1151] 10.222.0.23 () {58 vars in 1059 bytes} [Thu Oct 13 13:04:44 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:04:44.851278 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f73a59f-7c6a-460f-b298-7bb943655502 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:04:44.852777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-0f73a59f-7c6a-460f-b298-7bb943655502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:44.957640 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f73a59f-7c6a-460f-b298-7bb943655502 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:04:45.031074 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d645fd1-ac81-4869-afae-4c55cdf731e5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ce234c5b4643b2ac6341e0928f3830', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:04:45.031694 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d645fd1-ac81-4869-afae-4c55cdf731e5 service glance] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:45.031967 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d645fd1-ac81-4869-afae-4c55cdf731e5 service glance] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:45.032232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d645fd1-ac81-4869-afae-4c55cdf731e5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:45.033790 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-5d645fd1-ac81-4869-afae-4c55cdf731e5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:45.087797 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:04:45.141968 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d645fd1-ac81-4869-afae-4c55cdf731e5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:04:45.143431 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d645fd1-ac81-4869-afae-4c55cdf731e5 service glance] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:04:45.161798 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 574/1152] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:04:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:04:47.408418 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904601a9-33d4-4f8e-9eda-b89997c58d9d None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:47.409024 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904601a9-33d4-4f8e-9eda-b89997c58d9d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:47.409024 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904601a9-33d4-4f8e-9eda-b89997c58d9d None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:47.410312 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 579/1153] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:04:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:04:47.415625 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291f18f-f5ff-498e-86b6-a09ee1137e1b None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:47.416092 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291f18f-f5ff-498e-86b6-a09ee1137e1b None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:47.416201 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291f18f-f5ff-498e-86b6-a09ee1137e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:47.480606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-9291f18f-f5ff-498e-86b6-a09ee1137e1b None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:04:47.481803 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-9291f18f-f5ff-498e-86b6-a09ee1137e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:47.577712 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 575/1154] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:04:47.854412 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904601a9-33d4-4f8e-9eda-b89997c58d9d None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:04:47.856149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-904601a9-33d4-4f8e-9eda-b89997c58d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:47.980327 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904601a9-33d4-4f8e-9eda-b89997c58d9d None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:04:48.032513 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:04:48.076525 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5585a2d-ff4d-4cd4-8942-0b87a1532c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:04:48.077131 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5585a2d-ff4d-4cd4-8942-0b87a1532c22 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:48.077408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5585a2d-ff4d-4cd4-8942-0b87a1532c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:48.077673 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5585a2d-ff4d-4cd4-8942-0b87a1532c22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:48.078301 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5585a2d-ff4d-4cd4-8942-0b87a1532c22 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:04:48.079436 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: WARNING py.warnings [None req-e5585a2d-ff4d-4cd4-8942-0b87a1532c22 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 13 13:04:48.079436 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: warnings.warn(msg) Oct 13 13:04:48.079436 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: Oct 13 13:04:48.080573 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5585a2d-ff4d-4cd4-8942-0b87a1532c22 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:04:48.098964 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 580/1155] 10.222.0.23 () {56 vars in 1108 bytes} [Thu Oct 13 13:04:47 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:04:50.243770 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a79067-e5da-4fa5-bf8c-1326f6939d6c None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:50.244305 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a79067-e5da-4fa5-bf8c-1326f6939d6c None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:50.244305 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a79067-e5da-4fa5-bf8c-1326f6939d6c None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:50.245551 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 576/1156] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:04:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:04:50.250858 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff76c7-a2a9-4745-9413-ef4f279dd0f3 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:50.251285 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff76c7-a2a9-4745-9413-ef4f279dd0f3 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:50.251461 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff76c7-a2a9-4745-9413-ef4f279dd0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:50.315298 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-4bff76c7-a2a9-4745-9413-ef4f279dd0f3 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:04:50.316486 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4bff76c7-a2a9-4745-9413-ef4f279dd0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:50.408711 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 581/1157] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:04:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:04:50.701133 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a79067-e5da-4fa5-bf8c-1326f6939d6c None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:04:50.702636 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-66a79067-e5da-4fa5-bf8c-1326f6939d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:50.712805 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:04:50.846129 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a79067-e5da-4fa5-bf8c-1326f6939d6c None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:04:50.928342 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:04:50.961480 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9504f-a186-418e-a683-eaf83f6c9aec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:04:50.962091 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9504f-a186-418e-a683-eaf83f6c9aec service nova] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:50.962375 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9504f-a186-418e-a683-eaf83f6c9aec service nova] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:50.962648 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9504f-a186-418e-a683-eaf83f6c9aec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:50.964266 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-80d9504f-a186-418e-a683-eaf83f6c9aec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:51.052212 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9504f-a186-418e-a683-eaf83f6c9aec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:04:51.054366 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9504f-a186-418e-a683-eaf83f6c9aec service nova] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:04:51.072285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 577/1158] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:04:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:04:53.403369 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda8f9b-cca5-4a39-941b-0478f85d103a None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:53.403930 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda8f9b-cca5-4a39-941b-0478f85d103a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:53.403930 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda8f9b-cca5-4a39-941b-0478f85d103a None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:53.405249 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 582/1159] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:04:53.410610 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b0220-8a2f-48b7-a678-600e30f6e232 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:53.411071 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b0220-8a2f-48b7-a678-600e30f6e232 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:53.411188 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b0220-8a2f-48b7-a678-600e30f6e232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:53.476209 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-062b0220-8a2f-48b7-a678-600e30f6e232 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:04:53.477407 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-062b0220-8a2f-48b7-a678-600e30f6e232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:53.524271 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:04:53.573959 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 578/1160] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:04:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:04:53.856062 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda8f9b-cca5-4a39-941b-0478f85d103a None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:04:53.857614 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ecda8f9b-cca5-4a39-941b-0478f85d103a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:53.881733 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:04:53.959718 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda8f9b-cca5-4a39-941b-0478f85d103a None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:04:54.033129 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d55a4c6-14b9-4167-bddf-bc1ce6972bda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:04:54.033729 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55a4c6-14b9-4167-bddf-bc1ce6972bda service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:54.034007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55a4c6-14b9-4167-bddf-bc1ce6972bda service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:54.034276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d55a4c6-14b9-4167-bddf-bc1ce6972bda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:54.035930 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0d55a4c6-14b9-4167-bddf-bc1ce6972bda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:54.109376 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:04:54.143853 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d55a4c6-14b9-4167-bddf-bc1ce6972bda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:04:54.145285 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d55a4c6-14b9-4167-bddf-bc1ce6972bda service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:04:54.163690 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 583/1161] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:04:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:04:56.352685 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b90e1-ba26-4333-9dd1-12e0749fbb31 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:56.353228 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b90e1-ba26-4333-9dd1-12e0749fbb31 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:56.353228 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b90e1-ba26-4333-9dd1-12e0749fbb31 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:56.354584 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 579/1162] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:04:56.359789 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699a417-46a2-439d-923f-5c4fe8303f80 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:56.360177 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699a417-46a2-439d-923f-5c4fe8303f80 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:56.360336 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699a417-46a2-439d-923f-5c4fe8303f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:56.424286 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f699a417-46a2-439d-923f-5c4fe8303f80 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:04:56.425468 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f699a417-46a2-439d-923f-5c4fe8303f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:56.519013 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 584/1163] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:04:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:04:56.805266 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b90e1-ba26-4333-9dd1-12e0749fbb31 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:04:56.807047 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-8f6b90e1-ba26-4333-9dd1-12e0749fbb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:56.908198 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b90e1-ba26-4333-9dd1-12e0749fbb31 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:04:56.983163 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:04:56.984630 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14bf91-6977-4a3b-86c4-a453371062cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:04:56.985271 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14bf91-6977-4a3b-86c4-a453371062cf service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:56.985743 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14bf91-6977-4a3b-86c4-a453371062cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:56.986196 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14bf91-6977-4a3b-86c4-a453371062cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:56.987990 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-5b14bf91-6977-4a3b-86c4-a453371062cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:57.095315 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14bf91-6977-4a3b-86c4-a453371062cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:04:57.096783 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14bf91-6977-4a3b-86c4-a453371062cf service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:04:57.114645 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 580/1164] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:04:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:04:59.371698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba5fb9-d513-4212-bb4d-9049402c0b7e None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:59.372241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba5fb9-d513-4212-bb4d-9049402c0b7e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:59.372241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bba5fb9-d513-4212-bb4d-9049402c0b7e None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:59.373616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 585/1165] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:04:59.378946 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c802920-bae9-4df4-b768-d4383cbd9b8c None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:04:59.379389 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c802920-bae9-4df4-b768-d4383cbd9b8c None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:04:59.379530 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c802920-bae9-4df4-b768-d4383cbd9b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:04:59.443863 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-1c802920-bae9-4df4-b768-d4383cbd9b8c None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:04:59.445055 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-1c802920-bae9-4df4-b768-d4383cbd9b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:59.496442 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:04:59.540615 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 581/1166] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:04:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:04:59.831530 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bba5fb9-d513-4212-bb4d-9049402c0b7e None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:04:59.833077 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8bba5fb9-d513-4212-bb4d-9049402c0b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:04:59.868183 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:04:59.934924 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bba5fb9-d513-4212-bb4d-9049402c0b7e None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:05:00.007083 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff8e0ee-9ba4-47cb-ab77-e9ae6b92bcdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:05:00.007702 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8e0ee-9ba4-47cb-ab77-e9ae6b92bcdd service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:05:00.007977 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8e0ee-9ba4-47cb-ab77-e9ae6b92bcdd service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:05:00.008245 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8e0ee-9ba4-47cb-ab77-e9ae6b92bcdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:05:00.009831 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2ff8e0ee-9ba4-47cb-ab77-e9ae6b92bcdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:05:00.091857 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:05:00.119962 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff8e0ee-9ba4-47cb-ab77-e9ae6b92bcdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:05:00.121385 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff8e0ee-9ba4-47cb-ab77-e9ae6b92bcdd service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:05:00.139694 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 586/1167] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:04:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:05:02.526567 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927b279-a4e2-4c6d-9ecf-1a5864083822 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:05:02.527157 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927b279-a4e2-4c6d-9ecf-1a5864083822 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:05:02.527157 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927b279-a4e2-4c6d-9ecf-1a5864083822 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:05:02.528481 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 582/1168] 10.222.0.23 () {54 vars in 927 bytes} [Thu Oct 13 13:05:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 13 13:05:02.534006 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b8cd1-e4c2-4e18-92f4-e3ad6d889272 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:05:02.534425 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b8cd1-e4c2-4e18-92f4-e3ad6d889272 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:05:02.534588 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b8cd1-e4c2-4e18-92f4-e3ad6d889272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:05:02.599744 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-549b8cd1-e4c2-4e18-92f4-e3ad6d889272 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:05:02.601289 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-549b8cd1-e4c2-4e18-92f4-e3ad6d889272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:05:02.699631 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 587/1169] 10.222.0.23 () {58 vars in 1024 bytes} [Thu Oct 13 13:05:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:05:02.986524 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927b279-a4e2-4c6d-9ecf-1a5864083822 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:05:02.988185 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-f927b279-a4e2-4c6d-9ecf-1a5864083822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:05:03.088481 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927b279-a4e2-4c6d-9ecf-1a5864083822 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:05:03.159582 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5096c70-5b1d-4673-b62f-e92ebc7cfa90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:05:03.160224 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5096c70-5b1d-4673-b62f-e92ebc7cfa90 service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:05:03.160451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5096c70-5b1d-4673-b62f-e92ebc7cfa90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:05:03.160717 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5096c70-5b1d-4673-b62f-e92ebc7cfa90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:05:03.162835 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-e5096c70-5b1d-4673-b62f-e92ebc7cfa90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:05:03.172032 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:05:03.268066 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5096c70-5b1d-4673-b62f-e92ebc7cfa90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:05:03.269486 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5096c70-5b1d-4673-b62f-e92ebc7cfa90 service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:05:03.288850 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 583/1170] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:05:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:05:12.698091 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=82 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:05:13.285639 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=50 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:06:18.865083 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0b996-5418-4391-aa96-f7880d79c656 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:06:18.865625 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0b996-5418-4391-aa96-f7880d79c656 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:06:18.865625 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0b996-5418-4391-aa96-f7880d79c656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:06:18.943549 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0bf0b996-5418-4391-aa96-f7880d79c656 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:06:18.944749 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0bf0b996-5418-4391-aa96-f7880d79c656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:06:19.038585 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 588/1171] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:06:19.047138 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca2157-6dcc-4f77-89dc-37567415be4d None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:06:19.047660 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca2157-6dcc-4f77-89dc-37567415be4d None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:06:19.047765 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca2157-6dcc-4f77-89dc-37567415be4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:06:19.122691 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-cbca2157-6dcc-4f77-89dc-37567415be4d None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:06:19.123917 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-cbca2157-6dcc-4f77-89dc-37567415be4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:06:19.220574 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 584/1172] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:06:19.225272 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0b996-5418-4391-aa96-f7880d79c656 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:06:19.226534 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0bf0b996-5418-4391-aa96-f7880d79c656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:06:19.237814 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0b996-5418-4391-aa96-f7880d79c656 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:06:19.304825 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:06:19.337768 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82acbe73-aee4-4906-abdf-153870e009e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:06:19.338371 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82acbe73-aee4-4906-abdf-153870e009e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:06:19.338654 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82acbe73-aee4-4906-abdf-153870e009e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:06:19.338929 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82acbe73-aee4-4906-abdf-153870e009e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:06:19.339602 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82acbe73-aee4-4906-abdf-153870e009e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:06:19.339911 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82acbe73-aee4-4906-abdf-153870e009e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:06:19.340917 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: WARNING py.warnings [None req-82acbe73-aee4-4906-abdf-153870e009e2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 13 13:06:19.340917 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: warnings.warn(msg) Oct 13 13:06:19.340917 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: Oct 13 13:06:19.341808 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82acbe73-aee4-4906-abdf-153870e009e2 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:06:19.353275 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 589/1173] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:06:19.362597 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca2157-6dcc-4f77-89dc-37567415be4d None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:06:19.371680 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca2157-6dcc-4f77-89dc-37567415be4d None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:06:19.432477 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:06:19.466129 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d527167-c696-4500-afd0-73bb4355b0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:06:19.466724 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d527167-c696-4500-afd0-73bb4355b0bd admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:06:19.467003 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d527167-c696-4500-afd0-73bb4355b0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:06:19.467369 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d527167-c696-4500-afd0-73bb4355b0bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:06:19.468003 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d527167-c696-4500-afd0-73bb4355b0bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:06:19.469108 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: WARNING py.warnings [None req-6d527167-c696-4500-afd0-73bb4355b0bd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 13 13:06:19.469108 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: warnings.warn(msg) Oct 13 13:06:19.469108 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: Oct 13 13:06:19.470301 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d527167-c696-4500-afd0-73bb4355b0bd admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:06:19.524587 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 585/1174] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:06:19 2022] POST /identity/v3/projects => generated 379 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:06:19.528770 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82acbe73-aee4-4906-abdf-153870e009e2 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:06:19.530299 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-82acbe73-aee4-4906-abdf-153870e009e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:06:19.562998 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82acbe73-aee4-4906-abdf-153870e009e2 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:06:19.655699 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f7d40-6cad-48c0-9385-4a02ddba9178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:06:19.656303 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f7d40-6cad-48c0-9385-4a02ddba9178 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:06:19.656581 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f7d40-6cad-48c0-9385-4a02ddba9178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:06:19.656854 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f7d40-6cad-48c0-9385-4a02ddba9178 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:06:19.657489 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333f7d40-6cad-48c0-9385-4a02ddba9178 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:06:19.658491 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: WARNING py.warnings [None req-333f7d40-6cad-48c0-9385-4a02ddba9178 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 13 13:06:19.658491 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: warnings.warn(msg) Oct 13 13:06:19.658491 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: Oct 13 13:06:19.659723 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333f7d40-6cad-48c0-9385-4a02ddba9178 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:06:19.701610 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 590/1175] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:06:19 2022] POST /identity/v3/users => generated 290 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:06:19.705909 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d527167-c696-4500-afd0-73bb4355b0bd admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:06:19.707432 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-6d527167-c696-4500-afd0-73bb4355b0bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:06:19.767360 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d527167-c696-4500-afd0-73bb4355b0bd admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:06:19.848465 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:06:19.860019 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768b16d4-9d98-4393-8f0b-344fca0cd078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:06:19.860621 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b16d4-9d98-4393-8f0b-344fca0cd078 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:06:19.860900 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b16d4-9d98-4393-8f0b-344fca0cd078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:06:19.861172 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b16d4-9d98-4393-8f0b-344fca0cd078 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:06:19.861843 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768b16d4-9d98-4393-8f0b-344fca0cd078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:06:19.863474 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768b16d4-9d98-4393-8f0b-344fca0cd078 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:06:19.876306 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 586/1176] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:06:19.880305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f7d40-6cad-48c0-9385-4a02ddba9178 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:06:19.881811 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-333f7d40-6cad-48c0-9385-4a02ddba9178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:06:19.893681 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f7d40-6cad-48c0-9385-4a02ddba9178 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:06:19.918722 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:06:19.988261 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868327ea-88e3-42a9-a5f9-92e5eff9f5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:06:19.988868 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868327ea-88e3-42a9-a5f9-92e5eff9f5ac admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:06:19.989150 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868327ea-88e3-42a9-a5f9-92e5eff9f5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:06:19.989422 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868327ea-88e3-42a9-a5f9-92e5eff9f5ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:06:19.990062 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868327ea-88e3-42a9-a5f9-92e5eff9f5ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:06:19.992290 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868327ea-88e3-42a9-a5f9-92e5eff9f5ac admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:06:20.004403 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 591/1177] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:06:20.008549 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768b16d4-9d98-4393-8f0b-344fca0cd078 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:06:20.010067 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-768b16d4-9d98-4393-8f0b-344fca0cd078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:06:20.042597 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768b16d4-9d98-4393-8f0b-344fca0cd078 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:06:20.134580 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bfb594-e05d-4913-9f55-8db719750c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:06:20.135226 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bfb594-e05d-4913-9f55-8db719750c5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:06:20.135448 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bfb594-e05d-4913-9f55-8db719750c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:06:20.135715 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bfb594-e05d-4913-9f55-8db719750c5a admin admin] PATH_INFO: `/v3/projects/d5b161306ad14882be7e59aadf4ba8b6/users/21362e35d4f340bd9b00415824e4078e/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:06:20.157454 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bfb594-e05d-4913-9f55-8db719750c5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5b161306ad14882be7e59aadf4ba8b6, user_id=21362e35d4f340bd9b00415824e4078e, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:06:20.160111 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bfb594-e05d-4913-9f55-8db719750c5a admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:06:20.185668 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 587/1178] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:06:20 2022] PUT /identity/v3/projects/d5b161306ad14882be7e59aadf4ba8b6/users/21362e35d4f340bd9b00415824e4078e/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:06:20.196670 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d36b0-ad7a-4c22-a1ad-1bae62da5cf9 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:06:20.196961 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d36b0-ad7a-4c22-a1ad-1bae62da5cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:06:20.197240 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d36b0-ad7a-4c22-a1ad-1bae62da5cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:06:20.253668 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-436d36b0-ad7a-4c22-a1ad-1bae62da5cf9 None None] MFA Rules not processed for user `21362e35d4f340bd9b00415824e4078e`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:06:20.254875 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-436d36b0-ad7a-4c22-a1ad-1bae62da5cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:06:20.330229 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 592/1179] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:06:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:06:20.343082 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476f3d7-df35-4b4e-8e7a-826ebd3fd5e2 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:06:20.343376 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476f3d7-df35-4b4e-8e7a-826ebd3fd5e2 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:06:20.343649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476f3d7-df35-4b4e-8e7a-826ebd3fd5e2 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:06:20.344972 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 588/1180] 10.222.0.23 () {54 vars in 963 bytes} [Thu Oct 13 13:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:06:20.349859 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948fc28-35e5-43a6-9800-170e238a4600 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:06:20.350154 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948fc28-35e5-43a6-9800-170e238a4600 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:06:20.350434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948fc28-35e5-43a6-9800-170e238a4600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:06:20.381502 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:06:20.424496 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-a948fc28-35e5-43a6-9800-170e238a4600 None None] MFA Rules not processed for user `2398a599165a46adb5718d98c897f238`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:06:20.425640 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a948fc28-35e5-43a6-9800-170e238a4600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:06:20.500824 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 593/1181] 10.222.0.23 () {58 vars in 1060 bytes} [Thu Oct 13 13:06:20 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:06:20.509121 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1476f3d7-df35-4b4e-8e7a-826ebd3fd5e2 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:06:20.510606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-1476f3d7-df35-4b4e-8e7a-826ebd3fd5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:06:20.560188 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:06:20.612948 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1476f3d7-df35-4b4e-8e7a-826ebd3fd5e2 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:06:20.685265 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d44ab-8988-4c02-bd55-f3a0694ea7c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:06:20.685794 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d44ab-8988-4c02-bd55-f3a0694ea7c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:06:20.686067 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d44ab-8988-4c02-bd55-f3a0694ea7c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:06:20.686333 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d44ab-8988-4c02-bd55-f3a0694ea7c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:06:20.687948 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-689d44ab-8988-4c02-bd55-f3a0694ea7c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:06:20.782944 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d44ab-8988-4c02-bd55-f3a0694ea7c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:06:20.784468 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d44ab-8988-4c02-bd55-f3a0694ea7c4 service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:06:20.787824 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:06:20.802882 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 589/1182] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:06:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:06:20.848379 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49324bc-b7b1-4075-a48a-7db25fef9d72 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:06:20.848671 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49324bc-b7b1-4075-a48a-7db25fef9d72 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:06:20.848950 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49324bc-b7b1-4075-a48a-7db25fef9d72 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:06:20.850248 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 594/1183] 10.222.0.23 () {52 vars in 830 bytes} [Thu Oct 13 13:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:06:20.854436 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d44ab-8988-4c02-bd55-f3a0694ea7c4 service cinder] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:06:20.855658 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-689d44ab-8988-4c02-bd55-f3a0694ea7c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:06:20.953636 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d44ab-8988-4c02-bd55-f3a0694ea7c4 service cinder] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:06:21.045257 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943d1b4-4816-4dc5-a2c1-bfa00662d549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:06:21.045871 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943d1b4-4816-4dc5-a2c1-bfa00662d549 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:06:21.046146 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943d1b4-4816-4dc5-a2c1-bfa00662d549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:06:21.046413 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943d1b4-4816-4dc5-a2c1-bfa00662d549 admin admin] PATH_INFO: `/v3/users/21362e35d4f340bd9b00415824e4078e` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:06:21.058864 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0943d1b4-4816-4dc5-a2c1-bfa00662d549 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21362e35d4f340bd9b00415824e4078e)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:06:21.059924 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: WARNING py.warnings [None req-0943d1b4-4816-4dc5-a2c1-bfa00662d549 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 13 13:06:21.059924 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: warnings.warn(msg) Oct 13 13:06:21.059924 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: Oct 13 13:06:21.061126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0943d1b4-4816-4dc5-a2c1-bfa00662d549 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:06:21.070639 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:06:21.125300 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-0943d1b4-4816-4dc5-a2c1-bfa00662d549 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21362e35d4f340bd9b00415824e4078e'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:06:21.139314 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-0943d1b4-4816-4dc5-a2c1-bfa00662d549 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21362e35d4f340bd9b00415824e4078e'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:06:21.139903 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-0943d1b4-4816-4dc5-a2c1-bfa00662d549 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21362e35d4f340bd9b00415824e4078e'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:06:21.140278 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-0943d1b4-4816-4dc5-a2c1-bfa00662d549 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21362e35d4f340bd9b00415824e4078e'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:06:21.149947 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-0943d1b4-4816-4dc5-a2c1-bfa00662d549 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21362e35d4f340bd9b00415824e4078e'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:06:21.179125 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 590/1184] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:06:20 2022] DELETE /identity/v3/users/21362e35d4f340bd9b00415824e4078e => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:06:21.183730 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18196cbe-832c-4118-84dd-578e64bc070c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:06:21.184218 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18196cbe-832c-4118-84dd-578e64bc070c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:06:21.184327 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18196cbe-832c-4118-84dd-578e64bc070c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:06:21.240943 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-18196cbe-832c-4118-84dd-578e64bc070c None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:06:21.242078 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-18196cbe-832c-4118-84dd-578e64bc070c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:06:21.314729 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:06:21.327568 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 595/1185] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:06:21.337664 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943d1b4-4816-4dc5-a2c1-bfa00662d549 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:06:21.338894 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-0943d1b4-4816-4dc5-a2c1-bfa00662d549 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:06:21.434992 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943d1b4-4816-4dc5-a2c1-bfa00662d549 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:06:21.487444 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:06:21.506956 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11795b4-b863-4146-8610-8e58a55c32eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:06:21.508055 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11795b4-b863-4146-8610-8e58a55c32eb service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:06:21.508330 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11795b4-b863-4146-8610-8e58a55c32eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:06:21.508598 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11795b4-b863-4146-8610-8e58a55c32eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:06:21.510242 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-c11795b4-b863-4146-8610-8e58a55c32eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:06:21.616530 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11795b4-b863-4146-8610-8e58a55c32eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:06:21.617966 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11795b4-b863-4146-8610-8e58a55c32eb service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:06:21.636190 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 591/1186] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:06:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:06:21.974280 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18196cbe-832c-4118-84dd-578e64bc070c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:06:21.975528 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-18196cbe-832c-4118-84dd-578e64bc070c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:06:21.987602 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18196cbe-832c-4118-84dd-578e64bc070c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:06:22.082124 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2c3147-413d-4ee8-91c4-0f3fe44d337a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:06:22.082775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c3147-413d-4ee8-91c4-0f3fe44d337a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:06:22.083062 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c3147-413d-4ee8-91c4-0f3fe44d337a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:06:22.083352 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c3147-413d-4ee8-91c4-0f3fe44d337a admin admin] PATH_INFO: `/v3/projects/d5b161306ad14882be7e59aadf4ba8b6` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:06:22.095220 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2c3147-413d-4ee8-91c4-0f3fe44d337a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5b161306ad14882be7e59aadf4ba8b6)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:06:22.096236 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: WARNING py.warnings [None req-ff2c3147-413d-4ee8-91c4-0f3fe44d337a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 13 13:06:22.096236 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: warnings.warn(msg) Oct 13 13:06:22.096236 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: Oct 13 13:06:22.097479 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2c3147-413d-4ee8-91c4-0f3fe44d337a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:06:22.153953 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ff2c3147-413d-4ee8-91c4-0f3fe44d337a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5b161306ad14882be7e59aadf4ba8b6', 'request_id': 'req-ff2c3147-413d-4ee8-91c4-0f3fe44d337a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:06:22.159924 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ff2c3147-413d-4ee8-91c4-0f3fe44d337a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5b161306ad14882be7e59aadf4ba8b6', 'request_id': 'req-ff2c3147-413d-4ee8-91c4-0f3fe44d337a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:06:22.172563 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ff2c3147-413d-4ee8-91c4-0f3fe44d337a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5b161306ad14882be7e59aadf4ba8b6', 'request_id': 'req-ff2c3147-413d-4ee8-91c4-0f3fe44d337a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:06:22.199123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ff2c3147-413d-4ee8-91c4-0f3fe44d337a admin admin] The token cache is being invalidate because project d5b161306ad14882be7e59aadf4ba8b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:06:22.199724 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ff2c3147-413d-4ee8-91c4-0f3fe44d337a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:06:22.199870 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ff2c3147-413d-4ee8-91c4-0f3fe44d337a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:06:22.201183 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 596/1187] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:06:21 2022] DELETE /identity/v3/projects/d5b161306ad14882be7e59aadf4ba8b6 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:06:31.633974 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=62 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:06:32.197129 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:07:49.718785 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af179a-bfd9-469e-be9f-9f9c39ac7a8b None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:07:49.719452 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af179a-bfd9-469e-be9f-9f9c39ac7a8b None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:07:49.719452 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af179a-bfd9-469e-be9f-9f9c39ac7a8b None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:07:49.720811 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 592/1188] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 13 13:07:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:07:49.725876 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ca3d2-572a-495b-a1d7-0be819a85a17 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:07:49.726296 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ca3d2-572a-495b-a1d7-0be819a85a17 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:07:49.726402 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ca3d2-572a-495b-a1d7-0be819a85a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:07:49.805983 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d30ca3d2-572a-495b-a1d7-0be819a85a17 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:07:49.807238 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d30ca3d2-572a-495b-a1d7-0be819a85a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:07:49.907030 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 597/1189] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 13 13:07:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:07:50.060791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af179a-bfd9-469e-be9f-9f9c39ac7a8b None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:07:50.062889 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-25af179a-bfd9-469e-be9f-9f9c39ac7a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:07:50.180501 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af179a-bfd9-469e-be9f-9f9c39ac7a8b None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:07:50.186334 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7deafc1-65c7-4c58-ad4f-3112a22458fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:07:50.186926 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7deafc1-65c7-4c58-ad4f-3112a22458fa service nova] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:07:50.187235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7deafc1-65c7-4c58-ad4f-3112a22458fa service nova] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:07:50.187506 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7deafc1-65c7-4c58-ad4f-3112a22458fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:07:50.195299 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7deafc1-65c7-4c58-ad4f-3112a22458fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:07:50.196880 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7deafc1-65c7-4c58-ad4f-3112a22458fa service nova] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:07:50.206487 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 593/1190] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:07:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:07:51.321539 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a930f9-81da-4df0-87e6-953b24a83295 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:07:51.322169 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a930f9-81da-4df0-87e6-953b24a83295 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:07:51.322285 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a930f9-81da-4df0-87e6-953b24a83295 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:07:51.324224 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 598/1191] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 13 13:07:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:07:51.331214 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b2c1c-4e02-40a1-9b78-4c09e2fb2226 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:07:51.331707 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b2c1c-4e02-40a1-9b78-4c09e2fb2226 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:07:51.331707 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b2c1c-4e02-40a1-9b78-4c09e2fb2226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:07:51.350907 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-697b2c1c-4e02-40a1-9b78-4c09e2fb2226 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:07:51.352374 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-697b2c1c-4e02-40a1-9b78-4c09e2fb2226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:07:51.360210 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 594/1192] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 13 13:07:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:07:51.510559 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a930f9-81da-4df0-87e6-953b24a83295 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:07:51.525130 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a930f9-81da-4df0-87e6-953b24a83295 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:07:51.531194 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1e0ed-98fd-4bd4-b89f-5a8d288646a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:07:51.531787 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1e0ed-98fd-4bd4-b89f-5a8d288646a8 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:07:51.532072 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1e0ed-98fd-4bd4-b89f-5a8d288646a8 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:07:51.532348 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1e0ed-98fd-4bd4-b89f-5a8d288646a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:07:51.540202 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1e0ed-98fd-4bd4-b89f-5a8d288646a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:07:51.541782 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1e0ed-98fd-4bd4-b89f-5a8d288646a8 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:07:51.551271 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 599/1193] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:07:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:07:51.865662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6cc467-01f7-4bf4-a6b7-4aedb0f5a718 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:07:51.865875 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6cc467-01f7-4bf4-a6b7-4aedb0f5a718 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:07:51.866121 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6cc467-01f7-4bf4-a6b7-4aedb0f5a718 None None] PATH_INFO: `/v3` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:07:51.867444 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 595/1194] 10.222.0.50 () {54 vars in 925 bytes} [Thu Oct 13 13:07:51 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:07:51.873676 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e399a-4ebc-422d-8a17-dd35698800d1 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:07:51.873965 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e399a-4ebc-422d-8a17-dd35698800d1 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:07:51.874240 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e399a-4ebc-422d-8a17-dd35698800d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:07:51.935264 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-192e399a-4ebc-422d-8a17-dd35698800d1 None None] MFA Rules not processed for user `1a971cdb7f9d4e6692c6326c103d7303`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:07:51.936470 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-192e399a-4ebc-422d-8a17-dd35698800d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:07:51.953021 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:07:52.006555 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 600/1195] 10.222.0.50 () {58 vars in 1015 bytes} [Thu Oct 13 13:07:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:07:55.512039 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f7d1d-9855-4a94-83bf-828b09712824 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:07:55.512510 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f7d1d-9855-4a94-83bf-828b09712824 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:07:55.512618 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f7d1d-9855-4a94-83bf-828b09712824 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:07:55.513940 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 596/1196] 10.222.0.23 () {54 vars in 961 bytes} [Thu Oct 13 13:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:07:55.518751 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377ffa0-54f9-4ed4-bb8b-c1c24f5067b3 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:07:55.519215 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377ffa0-54f9-4ed4-bb8b-c1c24f5067b3 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:07:55.519323 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377ffa0-54f9-4ed4-bb8b-c1c24f5067b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:07:55.538220 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-2377ffa0-54f9-4ed4-bb8b-c1c24f5067b3 None None] MFA Rules not processed for user `1a971cdb7f9d4e6692c6326c103d7303`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:07:55.539436 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2377ffa0-54f9-4ed4-bb8b-c1c24f5067b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:07:55.547669 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 601/1197] 10.222.0.23 () {58 vars in 1058 bytes} [Thu Oct 13 13:07:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:07:55.555112 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f7d1d-9855-4a94-83bf-828b09712824 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:07:55.568761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f7d1d-9855-4a94-83bf-828b09712824 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:07:55.574352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d0e4b-26d7-48d0-9339-6c20192ef23c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a971cdb7f9d4e6692c6326c103d7303', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:07:55.574942 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d0e4b-26d7-48d0-9339-6c20192ef23c service placement] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:07:55.575235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d0e4b-26d7-48d0-9339-6c20192ef23c service placement] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:07:55.575504 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d0e4b-26d7-48d0-9339-6c20192ef23c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:07:55.583435 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d0e4b-26d7-48d0-9339-6c20192ef23c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:07:55.585001 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d0e4b-26d7-48d0-9339-6c20192ef23c service placement] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:07:55.594249 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 597/1198] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:07:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:08:02.994976 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3259b-0e1c-4571-bcef-6e2e596989b8 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:08:02.995569 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3259b-0e1c-4571-bcef-6e2e596989b8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:08:02.995569 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3259b-0e1c-4571-bcef-6e2e596989b8 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:08:02.996884 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 602/1199] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 13 13:08:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:08:03.003296 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d10904-7614-4d8b-ba68-8e713ec1c323 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:08:03.003709 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d10904-7614-4d8b-ba68-8e713ec1c323 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:08:03.003844 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d10904-7614-4d8b-ba68-8e713ec1c323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:08:03.022750 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-25d10904-7614-4d8b-ba68-8e713ec1c323 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:08:03.023977 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-25d10904-7614-4d8b-ba68-8e713ec1c323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:08:03.032310 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 598/1200] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 13 13:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:08:03.186029 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd3259b-0e1c-4571-bcef-6e2e596989b8 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:08:03.200340 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd3259b-0e1c-4571-bcef-6e2e596989b8 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:08:03.206243 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022752c4-0e82-4875-961e-c1caaf83cab3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:08:03.206832 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022752c4-0e82-4875-961e-c1caaf83cab3 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:08:03.207150 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022752c4-0e82-4875-961e-c1caaf83cab3 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:08:03.207426 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022752c4-0e82-4875-961e-c1caaf83cab3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:08:03.215466 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022752c4-0e82-4875-961e-c1caaf83cab3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:08:03.217723 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022752c4-0e82-4875-961e-c1caaf83cab3 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:08:03.226991 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 603/1201] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:08:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:08:04.288227 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feebe0e-b157-427e-bcb1-592526e21fee None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:08:04.288784 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feebe0e-b157-427e-bcb1-592526e21fee None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:08:04.288784 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feebe0e-b157-427e-bcb1-592526e21fee None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:08:04.290065 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 599/1202] 127.0.0.1 () {54 vars in 901 bytes} [Thu Oct 13 13:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:08:04.296753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ce53d-a935-4075-8ed5-76430d2e94e0 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:08:04.297175 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ce53d-a935-4075-8ed5-76430d2e94e0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:08:04.297327 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ce53d-a935-4075-8ed5-76430d2e94e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:08:04.316850 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8e5ce53d-a935-4075-8ed5-76430d2e94e0 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:08:04.318030 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8e5ce53d-a935-4075-8ed5-76430d2e94e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:08:04.325613 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 604/1203] 127.0.0.1 () {58 vars in 998 bytes} [Thu Oct 13 13:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:08:04.476282 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feebe0e-b157-427e-bcb1-592526e21fee None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:08:04.490987 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feebe0e-b157-427e-bcb1-592526e21fee None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:08:04.496363 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e8655-2c96-4ca8-855c-f2bfb096b9d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:08:04.496946 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e8655-2c96-4ca8-855c-f2bfb096b9d0 service nova] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:08:04.497225 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e8655-2c96-4ca8-855c-f2bfb096b9d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:08:04.497492 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e8655-2c96-4ca8-855c-f2bfb096b9d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:08:04.505246 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1e8655-2c96-4ca8-855c-f2bfb096b9d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:08:04.506807 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1e8655-2c96-4ca8-855c-f2bfb096b9d0 service nova] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:08:04.516222 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 600/1204] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:08:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:08:08.459909 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd87ac5-1afa-4183-987a-63319e660e2d None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:08:08.460473 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd87ac5-1afa-4183-987a-63319e660e2d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:08:08.460473 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd87ac5-1afa-4183-987a-63319e660e2d None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:08:08.461799 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 605/1205] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 13 13:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:08:08.467790 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b89bcf-b4bd-488a-833a-0fa5ebeca14f None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:08:08.468205 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b89bcf-b4bd-488a-833a-0fa5ebeca14f None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:08:08.468361 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b89bcf-b4bd-488a-833a-0fa5ebeca14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:08:08.487596 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-31b89bcf-b4bd-488a-833a-0fa5ebeca14f None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:08:08.488830 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-31b89bcf-b4bd-488a-833a-0fa5ebeca14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:08:08.496466 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 601/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 13 13:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:08:08.788017 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd87ac5-1afa-4183-987a-63319e660e2d None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:08:08.790026 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-abd87ac5-1afa-4183-987a-63319e660e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:08:08.868344 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd87ac5-1afa-4183-987a-63319e660e2d None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:08:08.871836 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2f2ea-48b0-48e6-8921-f310cb9b1a56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:08:08.872957 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2f2ea-48b0-48e6-8921-f310cb9b1a56 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:08:08.873216 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2f2ea-48b0-48e6-8921-f310cb9b1a56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:08:08.873486 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2f2ea-48b0-48e6-8921-f310cb9b1a56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:08:08.882077 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d2f2ea-48b0-48e6-8921-f310cb9b1a56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:08:08.883531 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d2f2ea-48b0-48e6-8921-f310cb9b1a56 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:08:08.893410 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 606/1207] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:08:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:08:11.017728 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f2b1e-0929-4fe3-898b-900dd0350d59 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:08:11.018257 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f2b1e-0929-4fe3-898b-900dd0350d59 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:08:11.018257 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f2b1e-0929-4fe3-898b-900dd0350d59 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:08:11.019842 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 602/1208] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 13 13:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:08:11.025068 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a656a89-7f22-4cc8-be98-8ec531fad408 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:08:11.025494 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a656a89-7f22-4cc8-be98-8ec531fad408 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:08:11.025679 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a656a89-7f22-4cc8-be98-8ec531fad408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:08:11.046309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-1a656a89-7f22-4cc8-be98-8ec531fad408 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:08:11.047474 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1a656a89-7f22-4cc8-be98-8ec531fad408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:08:11.055877 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 607/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 13 13:08:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:08:11.341750 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f2b1e-0929-4fe3-898b-900dd0350d59 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:08:11.352254 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f2b1e-0929-4fe3-898b-900dd0350d59 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:08:11.355713 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f654a279-d16e-45cf-9851-5cac59b53c38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:08:11.356353 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654a279-d16e-45cf-9851-5cac59b53c38 service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:08:11.357174 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654a279-d16e-45cf-9851-5cac59b53c38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:08:11.357443 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654a279-d16e-45cf-9851-5cac59b53c38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:08:11.365962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f654a279-d16e-45cf-9851-5cac59b53c38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:08:11.367424 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f654a279-d16e-45cf-9851-5cac59b53c38 service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:08:11.376494 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 603/1210] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:08:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:08:15.182607 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc37d91-fb5e-4b11-a96a-05a45713a7ed None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:08:15.183194 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc37d91-fb5e-4b11-a96a-05a45713a7ed None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:08:15.183194 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc37d91-fb5e-4b11-a96a-05a45713a7ed None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:08:15.185107 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 608/1211] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 13 13:08:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:08:15.190305 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0399871f-2a71-4ee5-8331-e03ac7b89bd2 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:08:15.190750 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0399871f-2a71-4ee5-8331-e03ac7b89bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:08:15.190846 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0399871f-2a71-4ee5-8331-e03ac7b89bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:08:15.198383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=81 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:08:15.210937 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-0399871f-2a71-4ee5-8331-e03ac7b89bd2 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:08:15.212176 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-0399871f-2a71-4ee5-8331-e03ac7b89bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:08:15.220724 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 604/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 13 13:08:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:08:15.522825 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc37d91-fb5e-4b11-a96a-05a45713a7ed None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:08:15.524533 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1cc37d91-fb5e-4b11-a96a-05a45713a7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:08:15.587488 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:08:15.601274 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc37d91-fb5e-4b11-a96a-05a45713a7ed None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:08:15.604823 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6dddaf-59f9-4e86-bb8e-9b312bbd75ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:08:15.605423 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6dddaf-59f9-4e86-bb8e-9b312bbd75ad service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:08:15.605701 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6dddaf-59f9-4e86-bb8e-9b312bbd75ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:08:15.605973 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6dddaf-59f9-4e86-bb8e-9b312bbd75ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:08:15.615350 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6dddaf-59f9-4e86-bb8e-9b312bbd75ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:08:15.616771 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6dddaf-59f9-4e86-bb8e-9b312bbd75ad service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:08:15.626315 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 609/1213] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:08:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:08:17.738881 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b336f27-c615-4744-8e88-26f1d9340189 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:08:17.739484 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b336f27-c615-4744-8e88-26f1d9340189 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:08:17.739484 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b336f27-c615-4744-8e88-26f1d9340189 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:08:17.740807 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 605/1214] 127.0.0.1 () {54 vars in 917 bytes} [Thu Oct 13 13:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:08:17.745981 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25575ce-1c9a-4eea-9ca9-80cc4321d2fb None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:08:17.746417 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25575ce-1c9a-4eea-9ca9-80cc4321d2fb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:08:17.746545 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25575ce-1c9a-4eea-9ca9-80cc4321d2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:08:17.766752 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b25575ce-1c9a-4eea-9ca9-80cc4321d2fb None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:08:17.767997 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b25575ce-1c9a-4eea-9ca9-80cc4321d2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:08:17.776078 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 610/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Oct 13 13:08:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:08:18.073594 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b336f27-c615-4744-8e88-26f1d9340189 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:08:18.075319 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-6b336f27-c615-4744-8e88-26f1d9340189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:08:18.088253 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b336f27-c615-4744-8e88-26f1d9340189 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:08:18.091795 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1e3a3-2607-4881-9673-2bfa7a0ac348 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:08:18.092382 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1e3a3-2607-4881-9673-2bfa7a0ac348 service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:08:18.092660 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1e3a3-2607-4881-9673-2bfa7a0ac348 service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:08:18.092931 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1e3a3-2607-4881-9673-2bfa7a0ac348 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:08:18.101499 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1e3a3-2607-4881-9673-2bfa7a0ac348 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:08:18.102892 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1e3a3-2607-4881-9673-2bfa7a0ac348 service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:08:18.112471 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 606/1216] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:08:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:08:27.761410 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=11 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:08:28.108361 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=8 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:40.118473 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771ddb0-30a3-4140-9ef9-82150c7acba5 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:40.119738 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771ddb0-30a3-4140-9ef9-82150c7acba5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:40.119738 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771ddb0-30a3-4140-9ef9-82150c7acba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:40.144774 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0771ddb0-30a3-4140-9ef9-82150c7acba5 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:40.146137 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0771ddb0-30a3-4140-9ef9-82150c7acba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:40.159544 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 611/1217] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:40.170115 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02daf2db-7731-4fe6-8001-f28f986e90c2 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:40.170705 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02daf2db-7731-4fe6-8001-f28f986e90c2 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:40.170705 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02daf2db-7731-4fe6-8001-f28f986e90c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:40.197550 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-02daf2db-7731-4fe6-8001-f28f986e90c2 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:40.202329 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-02daf2db-7731-4fe6-8001-f28f986e90c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:40.214099 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 607/1218] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:40.219730 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771ddb0-30a3-4140-9ef9-82150c7acba5 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:40.232713 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0771ddb0-30a3-4140-9ef9-82150c7acba5 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:40.240339 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e39749f-c0f5-474b-9a96-8cab7e89cd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:40.240967 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39749f-c0f5-474b-9a96-8cab7e89cd4f admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:40.241246 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39749f-c0f5-474b-9a96-8cab7e89cd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:40.241517 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39749f-c0f5-474b-9a96-8cab7e89cd4f admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:40.242176 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e39749f-c0f5-474b-9a96-8cab7e89cd4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:09:40.242479 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e39749f-c0f5-474b-9a96-8cab7e89cd4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:40.244119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e39749f-c0f5-474b-9a96-8cab7e89cd4f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:40.260929 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 612/1219] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:09:40.269675 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02daf2db-7731-4fe6-8001-f28f986e90c2 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:40.285037 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02daf2db-7731-4fe6-8001-f28f986e90c2 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:40.292500 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a6ae9-ad16-4593-af91-d436c443ed52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:40.293311 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a6ae9-ad16-4593-af91-d436c443ed52 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:40.293724 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a6ae9-ad16-4593-af91-d436c443ed52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:40.294624 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a6ae9-ad16-4593-af91-d436c443ed52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:40.294742 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a6ae9-ad16-4593-af91-d436c443ed52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:40.296638 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a6ae9-ad16-4593-af91-d436c443ed52 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:40.361700 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 608/1220] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:09:40 2022] POST /identity/v3/projects => generated 371 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:09:40.366389 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e39749f-c0f5-474b-9a96-8cab7e89cd4f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:40.377655 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e39749f-c0f5-474b-9a96-8cab7e89cd4f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:40.458409 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56baa9ef-7dbb-4028-9574-2cf56acaa20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:40.458943 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56baa9ef-7dbb-4028-9574-2cf56acaa20f admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:40.459248 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56baa9ef-7dbb-4028-9574-2cf56acaa20f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:40.459519 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56baa9ef-7dbb-4028-9574-2cf56acaa20f admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:40.460167 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56baa9ef-7dbb-4028-9574-2cf56acaa20f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:40.462142 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56baa9ef-7dbb-4028-9574-2cf56acaa20f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:40.498534 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 613/1221] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:09:40 2022] POST /identity/v3/users => generated 286 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:09:40.502830 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a6ae9-ad16-4593-af91-d436c443ed52 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:40.520450 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a6ae9-ad16-4593-af91-d436c443ed52 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:40.527425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48297c43-cd19-4c8d-9d12-15dbc66a5df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:40.528050 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48297c43-cd19-4c8d-9d12-15dbc66a5df4 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:40.528325 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48297c43-cd19-4c8d-9d12-15dbc66a5df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:40.528596 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48297c43-cd19-4c8d-9d12-15dbc66a5df4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:40.529242 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48297c43-cd19-4c8d-9d12-15dbc66a5df4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:40.531212 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48297c43-cd19-4c8d-9d12-15dbc66a5df4 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:40.545100 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 609/1222] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:09:40.549558 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56baa9ef-7dbb-4028-9574-2cf56acaa20f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:40.564351 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56baa9ef-7dbb-4028-9574-2cf56acaa20f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:40.571223 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77161d-1442-40cf-bf60-bb36bd06e4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:40.571950 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77161d-1442-40cf-bf60-bb36bd06e4eb admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:40.572238 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77161d-1442-40cf-bf60-bb36bd06e4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:40.572510 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77161d-1442-40cf-bf60-bb36bd06e4eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:40.573883 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77161d-1442-40cf-bf60-bb36bd06e4eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:40.575715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77161d-1442-40cf-bf60-bb36bd06e4eb admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:40.592382 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 614/1223] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:09:40.596239 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48297c43-cd19-4c8d-9d12-15dbc66a5df4 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:40.609690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48297c43-cd19-4c8d-9d12-15dbc66a5df4 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:40.610234 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255d4e3-d51f-4a30-a656-39b25f3d444f None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:40.610632 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255d4e3-d51f-4a30-a656-39b25f3d444f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:40.610906 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255d4e3-d51f-4a30-a656-39b25f3d444f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:40.617577 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1493fca-9084-4fdb-b91d-fb764c093f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:40.620123 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1493fca-9084-4fdb-b91d-fb764c093f1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:40.620420 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1493fca-9084-4fdb-b91d-fb764c093f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:40.620698 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1493fca-9084-4fdb-b91d-fb764c093f1d admin admin] PATH_INFO: `/v3/projects/fcc66b8f231d4b47be0c6c6b4b52b389/users/8c83931f54cb4d16b530cf7b4b817899/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:40.646776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d255d4e3-d51f-4a30-a656-39b25f3d444f None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:40.648035 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d255d4e3-d51f-4a30-a656-39b25f3d444f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:40.652973 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1493fca-9084-4fdb-b91d-fb764c093f1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcc66b8f231d4b47be0c6c6b4b52b389, user_id=8c83931f54cb4d16b530cf7b4b817899, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:40.655657 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1493fca-9084-4fdb-b91d-fb764c093f1d admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:40.670736 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 615/1224] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:09:40.671917 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d57be-8650-4f8f-aebb-ba148b503eb0 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:40.673601 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d57be-8650-4f8f-aebb-ba148b503eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:40.673872 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d57be-8650-4f8f-aebb-ba148b503eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:40.682131 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 610/1225] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:09:40 2022] PUT /identity/v3/projects/fcc66b8f231d4b47be0c6c6b4b52b389/users/8c83931f54cb4d16b530cf7b4b817899/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:09:40.684012 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b25b8e-70e5-4559-bde5-183382470e2e None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:40.684296 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b25b8e-70e5-4559-bde5-183382470e2e None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:40.684567 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b25b8e-70e5-4559-bde5-183382470e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:40.729622 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c07d57be-8650-4f8f-aebb-ba148b503eb0 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:40.731562 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c07d57be-8650-4f8f-aebb-ba148b503eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:40.735879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-74b25b8e-70e5-4559-bde5-183382470e2e None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:40.738397 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-74b25b8e-70e5-4559-bde5-183382470e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:40.838170 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 616/1226] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:40.839171 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e19b57-7f70-4299-9700-f1250b37c6ae None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:40.839463 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e19b57-7f70-4299-9700-f1250b37c6ae None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:40.839737 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e19b57-7f70-4299-9700-f1250b37c6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:40.845033 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 611/1227] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:40.846602 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d03fc-7f10-45eb-b612-9dded7ad8dfb None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:40.847160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d03fc-7f10-45eb-b612-9dded7ad8dfb None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:40.847659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d03fc-7f10-45eb-b612-9dded7ad8dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:40.865912 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-56e19b57-7f70-4299-9700-f1250b37c6ae None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:40.867135 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-56e19b57-7f70-4299-9700-f1250b37c6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:40.875670 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 617/1228] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:40.879243 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8132a55-3193-40ae-aaa7-226df148940d None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:40.879243 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8132a55-3193-40ae-aaa7-226df148940d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:40.879243 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8132a55-3193-40ae-aaa7-226df148940d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:40.880104 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-f34d03fc-7f10-45eb-b612-9dded7ad8dfb None None] MFA Rules not processed for user `8c83931f54cb4d16b530cf7b4b817899`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:40.881584 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-f34d03fc-7f10-45eb-b612-9dded7ad8dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:40.896315 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d8132a55-3193-40ae-aaa7-226df148940d None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:40.897472 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d8132a55-3193-40ae-aaa7-226df148940d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:40.912150 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 618/1229] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:40.912150 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0330af-0d31-4e39-9f6a-6a6f22995d43 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:40.912150 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0330af-0d31-4e39-9f6a-6a6f22995d43 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:40.912150 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0330af-0d31-4e39-9f6a-6a6f22995d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:40.930652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-3a0330af-0d31-4e39-9f6a-6a6f22995d43 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:40.931900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3a0330af-0d31-4e39-9f6a-6a6f22995d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:40.940733 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 619/1230] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:40.942407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc0e42-d0a1-459e-9779-b0b848b94c87 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:40.942701 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc0e42-d0a1-459e-9779-b0b848b94c87 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:40.943122 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc0e42-d0a1-459e-9779-b0b848b94c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:40.952533 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 612/1231] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:40.953806 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9aa86-6323-4d69-a845-2d49259d2ca2 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:40.954093 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9aa86-6323-4d69-a845-2d49259d2ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:40.954364 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9aa86-6323-4d69-a845-2d49259d2ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:40.964011 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-1abc0e42-d0a1-459e-9779-b0b848b94c87 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:40.965214 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1abc0e42-d0a1-459e-9779-b0b848b94c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:40.973484 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 620/1232] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:40.973678 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-15f9aa86-6323-4d69-a845-2d49259d2ca2 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:40.975012 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-15f9aa86-6323-4d69-a845-2d49259d2ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:40.975157 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a15de3-c4a1-4060-89eb-5537d5c04232 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:40.975359 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a15de3-c4a1-4060-89eb-5537d5c04232 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:40.975762 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a15de3-c4a1-4060-89eb-5537d5c04232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:40.988068 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 613/1233] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:40.989228 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea7b16-0ac1-4bc4-b11e-968a8b11141f None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:40.989509 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea7b16-0ac1-4bc4-b11e-968a8b11141f None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:40.989781 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea7b16-0ac1-4bc4-b11e-968a8b11141f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:41.002587 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d2a15de3-c4a1-4060-89eb-5537d5c04232 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:41.004085 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d2a15de3-c4a1-4060-89eb-5537d5c04232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:41.010226 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-e2ea7b16-0ac1-4bc4-b11e-968a8b11141f None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:41.012080 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-e2ea7b16-0ac1-4bc4-b11e-968a8b11141f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:41.015136 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 621/1234] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:41.015136 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a15de3-c4a1-4060-89eb-5537d5c04232 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:41.020912 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 614/1235] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:41.022105 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2578a2-89b4-4510-a15c-a70274115ac7 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:41.022387 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2578a2-89b4-4510-a15c-a70274115ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:41.022659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2578a2-89b4-4510-a15c-a70274115ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:41.023198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a15de3-c4a1-4060-89eb-5537d5c04232 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:41.030444 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809aef79-03b7-4666-8c73-1b671c40f872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:41.031048 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809aef79-03b7-4666-8c73-1b671c40f872 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:41.031238 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:41.031346 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809aef79-03b7-4666-8c73-1b671c40f872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:41.031617 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809aef79-03b7-4666-8c73-1b671c40f872 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:41.032271 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809aef79-03b7-4666-8c73-1b671c40f872 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:09:41.032578 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809aef79-03b7-4666-8c73-1b671c40f872 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:41.034267 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809aef79-03b7-4666-8c73-1b671c40f872 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:41.042895 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-ab2578a2-89b4-4510-a15c-a70274115ac7 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:41.044110 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-ab2578a2-89b4-4510-a15c-a70274115ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:41.053852 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 622/1236] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:09:41.053852 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1393b13-cb44-4341-b438-059083d536a9 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:41.053852 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1393b13-cb44-4341-b438-059083d536a9 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:41.053852 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1393b13-cb44-4341-b438-059083d536a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:41.054320 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 615/1237] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:41.054320 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385530ff-11a5-4e90-9db6-d9a690ebf37c None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:41.054320 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385530ff-11a5-4e90-9db6-d9a690ebf37c None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:41.054320 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385530ff-11a5-4e90-9db6-d9a690ebf37c None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:41.055212 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 616/1238] 10.222.0.23 () {54 vars in 961 bytes} [Thu Oct 13 13:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:09:41.056050 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c74b6-8594-447a-83aa-7bf914f2710b None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:41.056325 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c74b6-8594-447a-83aa-7bf914f2710b None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:41.056591 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c74b6-8594-447a-83aa-7bf914f2710b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:41.067387 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:41.072740 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b1393b13-cb44-4341-b438-059083d536a9 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:41.074139 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b1393b13-cb44-4341-b438-059083d536a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:41.077603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-d60c74b6-8594-447a-83aa-7bf914f2710b None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:41.079407 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-d60c74b6-8594-447a-83aa-7bf914f2710b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:41.085546 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 623/1239] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:41.085546 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1393b13-cb44-4341-b438-059083d536a9 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:41.091643 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 617/1240] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:41.093692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1393b13-cb44-4341-b438-059083d536a9 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:41.096217 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c74b6-8594-447a-83aa-7bf914f2710b None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:41.099125 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11090e5-fae3-4e9e-a4c7-24cba6e695d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:41.099742 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11090e5-fae3-4e9e-a4c7-24cba6e695d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:41.100024 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11090e5-fae3-4e9e-a4c7-24cba6e695d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:41.100299 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11090e5-fae3-4e9e-a4c7-24cba6e695d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:41.100955 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11090e5-fae3-4e9e-a4c7-24cba6e695d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:09:41.101266 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11090e5-fae3-4e9e-a4c7-24cba6e695d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:41.102875 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11090e5-fae3-4e9e-a4c7-24cba6e695d0 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:41.108055 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c74b6-8594-447a-83aa-7bf914f2710b None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:41.112651 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda41ddd-caa0-4f89-8d20-714e36101022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:41.113249 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda41ddd-caa0-4f89-8d20-714e36101022 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:41.113526 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda41ddd-caa0-4f89-8d20-714e36101022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:41.113797 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda41ddd-caa0-4f89-8d20-714e36101022 admin admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:41.114442 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda41ddd-caa0-4f89-8d20-714e36101022 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:09:41.114742 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda41ddd-caa0-4f89-8d20-714e36101022 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:41.115805 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: WARNING py.warnings [None req-fda41ddd-caa0-4f89-8d20-714e36101022 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 13 13:09:41.115805 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: warnings.warn(msg) Oct 13 13:09:41.115805 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: Oct 13 13:09:41.116257 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 624/1241] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:09:41.117592 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f4bb6-a360-4a79-bbf7-a1a38ad475bb None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:41.117882 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f4bb6-a360-4a79-bbf7-a1a38ad475bb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:41.118161 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f4bb6-a360-4a79-bbf7-a1a38ad475bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:41.124280 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda41ddd-caa0-4f89-8d20-714e36101022 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:41.143473 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 618/1242] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:09:41.143473 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda41ddd-caa0-4f89-8d20-714e36101022 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:41.143760 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b08f4bb6-a360-4a79-bbf7-a1a38ad475bb None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:41.143760 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b08f4bb6-a360-4a79-bbf7-a1a38ad475bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:41.147404 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda41ddd-caa0-4f89-8d20-714e36101022 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:41.148276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 625/1243] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:41.149182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f4bb6-a360-4a79-bbf7-a1a38ad475bb None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:41.152367 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50024e71-33fa-46fa-8425-d6d8cf37774d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:41.152957 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50024e71-33fa-46fa-8425-d6d8cf37774d admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:41.153231 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50024e71-33fa-46fa-8425-d6d8cf37774d admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:41.153497 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50024e71-33fa-46fa-8425-d6d8cf37774d admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:41.154238 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50024e71-33fa-46fa-8425-d6d8cf37774d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:41.156186 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50024e71-33fa-46fa-8425-d6d8cf37774d admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:41.158785 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f4bb6-a360-4a79-bbf7-a1a38ad475bb None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:41.163374 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4a19c-1000-4fbb-b8d5-edf6ad659b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:41.163959 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4a19c-1000-4fbb-b8d5-edf6ad659b49 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:41.164236 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4a19c-1000-4fbb-b8d5-edf6ad659b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:41.164714 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4a19c-1000-4fbb-b8d5-edf6ad659b49 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:41.165144 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4a19c-1000-4fbb-b8d5-edf6ad659b49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:09:41.165737 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4a19c-1000-4fbb-b8d5-edf6ad659b49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:41.167621 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4a19c-1000-4fbb-b8d5-edf6ad659b49 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:41.180240 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 626/1244] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:09:41.181819 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923814df-84b1-4bc7-9b08-7759655d0a68 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:41.182115 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923814df-84b1-4bc7-9b08-7759655d0a68 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:41.182942 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923814df-84b1-4bc7-9b08-7759655d0a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:41.195685 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 619/1245] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:09:41 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:09:41.199652 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50024e71-33fa-46fa-8425-d6d8cf37774d admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:41.225919 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-923814df-84b1-4bc7-9b08-7759655d0a68 None None] MFA Rules not processed for user `12700d985a724595af630c81e63f86d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:41.227258 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-923814df-84b1-4bc7-9b08-7759655d0a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:41.264327 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50024e71-33fa-46fa-8425-d6d8cf37774d admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:41.269267 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68b7a1-03a4-4036-95fc-aa483eabf1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:41.269872 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68b7a1-03a4-4036-95fc-aa483eabf1b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:41.270155 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68b7a1-03a4-4036-95fc-aa483eabf1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:41.270430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68b7a1-03a4-4036-95fc-aa483eabf1b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:41.271092 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d68b7a1-03a4-4036-95fc-aa483eabf1b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:09:41.271399 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d68b7a1-03a4-4036-95fc-aa483eabf1b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:41.273635 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d68b7a1-03a4-4036-95fc-aa483eabf1b1 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:41.284797 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 620/1246] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:09:41.284797 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68b7a1-03a4-4036-95fc-aa483eabf1b1 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:41.295607 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 627/1247] 10.222.0.23 () {58 vars in 1058 bytes} [Thu Oct 13 13:09:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:41.296055 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68b7a1-03a4-4036-95fc-aa483eabf1b1 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:41.296484 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923814df-84b1-4bc7-9b08-7759655d0a68 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:41.300718 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e167375-5fd1-4956-9f1e-8e8575775494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:41.301369 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e167375-5fd1-4956-9f1e-8e8575775494 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:41.301718 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e167375-5fd1-4956-9f1e-8e8575775494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:41.302052 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e167375-5fd1-4956-9f1e-8e8575775494 admin admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:41.303042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e167375-5fd1-4956-9f1e-8e8575775494 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:09:41.303430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e167375-5fd1-4956-9f1e-8e8575775494 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:41.305071 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e167375-5fd1-4956-9f1e-8e8575775494 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:41.306611 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923814df-84b1-4bc7-9b08-7759655d0a68 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:41.311922 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4694c6-81bd-4253-b8c7-e3c241f7a2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:41.312519 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4694c6-81bd-4253-b8c7-e3c241f7a2bc admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:41.312803 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4694c6-81bd-4253-b8c7-e3c241f7a2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:41.313079 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4694c6-81bd-4253-b8c7-e3c241f7a2bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:41.313710 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4694c6-81bd-4253-b8c7-e3c241f7a2bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:41.314754 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: WARNING py.warnings [None req-0f4694c6-81bd-4253-b8c7-e3c241f7a2bc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 13 13:09:41.314754 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: warnings.warn(msg) Oct 13 13:09:41.314754 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: Oct 13 13:09:41.315966 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4694c6-81bd-4253-b8c7-e3c241f7a2bc admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:41.324790 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 621/1248] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:09:41.324790 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e167375-5fd1-4956-9f1e-8e8575775494 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:41.328952 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e167375-5fd1-4956-9f1e-8e8575775494 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:41.333803 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66729f-4682-47b7-9761-454288564383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:41.334496 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66729f-4682-47b7-9761-454288564383 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:41.334791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66729f-4682-47b7-9761-454288564383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:41.335084 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66729f-4682-47b7-9761-454288564383 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:41.335724 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e66729f-4682-47b7-9761-454288564383 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:41.337678 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e66729f-4682-47b7-9761-454288564383 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:41.345949 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 628/1249] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:09:41 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:09:41.345949 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4694c6-81bd-4253-b8c7-e3c241f7a2bc admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:41.367852 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 622/1250] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:09:41 2022] POST /identity/v3/projects => generated 389 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:09:41.367852 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66729f-4682-47b7-9761-454288564383 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:41.407764 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4694c6-81bd-4253-b8c7-e3c241f7a2bc admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:41.429595 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66729f-4682-47b7-9761-454288564383 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:41.434247 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a472cdb2-a95f-4db7-995e-bf2acfd121db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:41.434873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a472cdb2-a95f-4db7-995e-bf2acfd121db admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:41.435185 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a472cdb2-a95f-4db7-995e-bf2acfd121db admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:41.435456 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a472cdb2-a95f-4db7-995e-bf2acfd121db admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:41.435926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:41.436079 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a472cdb2-a95f-4db7-995e-bf2acfd121db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:41.437983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a472cdb2-a95f-4db7-995e-bf2acfd121db admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:41.459539 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:41.469926 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 623/1251] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:09:41 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:09:41.471022 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370c6ef-fce9-47c1-84aa-828fd9d4b994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:41.471210 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a472cdb2-a95f-4db7-995e-bf2acfd121db admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:41.472235 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370c6ef-fce9-47c1-84aa-828fd9d4b994 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:41.472524 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370c6ef-fce9-47c1-84aa-828fd9d4b994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:41.472802 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370c6ef-fce9-47c1-84aa-828fd9d4b994 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:41.473720 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370c6ef-fce9-47c1-84aa-828fd9d4b994 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:09:41.474036 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370c6ef-fce9-47c1-84aa-828fd9d4b994 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:41.477112 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370c6ef-fce9-47c1-84aa-828fd9d4b994 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:41.492948 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 629/1252] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:09:41.493743 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370c6ef-fce9-47c1-84aa-828fd9d4b994 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:41.504381 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370c6ef-fce9-47c1-84aa-828fd9d4b994 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:41.536973 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a472cdb2-a95f-4db7-995e-bf2acfd121db admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:41.541679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940438ad-dc41-4fc9-b1cb-7f8475ed2f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:41.542860 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940438ad-dc41-4fc9-b1cb-7f8475ed2f93 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:41.543163 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940438ad-dc41-4fc9-b1cb-7f8475ed2f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:41.543457 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940438ad-dc41-4fc9-b1cb-7f8475ed2f93 admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:41.544092 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940438ad-dc41-4fc9-b1cb-7f8475ed2f93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:41.545105 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: WARNING py.warnings [None req-940438ad-dc41-4fc9-b1cb-7f8475ed2f93 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 13 13:09:41.545105 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: warnings.warn(msg) Oct 13 13:09:41.545105 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: Oct 13 13:09:41.546358 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940438ad-dc41-4fc9-b1cb-7f8475ed2f93 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:41.562380 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445016ba-a110-4f2c-b05e-023f807dfc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:41.562988 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445016ba-a110-4f2c-b05e-023f807dfc7a admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:41.563291 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445016ba-a110-4f2c-b05e-023f807dfc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:41.563562 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445016ba-a110-4f2c-b05e-023f807dfc7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:41.564188 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445016ba-a110-4f2c-b05e-023f807dfc7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:41.566109 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445016ba-a110-4f2c-b05e-023f807dfc7a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:41.570477 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 624/1253] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:09:41 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:09:41.570477 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940438ad-dc41-4fc9-b1cb-7f8475ed2f93 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:41.598549 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 630/1254] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:09:41 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:09:41.598549 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445016ba-a110-4f2c-b05e-023f807dfc7a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:41.639794 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940438ad-dc41-4fc9-b1cb-7f8475ed2f93 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:41.656892 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445016ba-a110-4f2c-b05e-023f807dfc7a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:41.661714 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716e6f7-36e3-4b41-908c-878eddbdc512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:41.662342 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716e6f7-36e3-4b41-908c-878eddbdc512 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:41.662615 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716e6f7-36e3-4b41-908c-878eddbdc512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:41.662911 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716e6f7-36e3-4b41-908c-878eddbdc512 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:41.663571 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4716e6f7-36e3-4b41-908c-878eddbdc512 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:41.665496 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4716e6f7-36e3-4b41-908c-878eddbdc512 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:41.692081 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 631/1255] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:09:41 2022] POST /identity/v3/projects => generated 391 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:09:41.696016 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716e6f7-36e3-4b41-908c-878eddbdc512 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:41.703767 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4269b3-303e-4178-add4-236ae234e303 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:41.704387 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4269b3-303e-4178-add4-236ae234e303 service nova] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:41.704672 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4269b3-303e-4178-add4-236ae234e303 service nova] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:41.704943 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4269b3-303e-4178-add4-236ae234e303 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:41.729904 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:41.734762 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:41.758253 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716e6f7-36e3-4b41-908c-878eddbdc512 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:41.763153 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5f3a5-62d7-4e09-baf1-c3529d823c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:41.763313 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4269b3-303e-4178-add4-236ae234e303 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:41.763805 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5f3a5-62d7-4e09-baf1-c3529d823c99 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:41.764079 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5f3a5-62d7-4e09-baf1-c3529d823c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:41.764375 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5f3a5-62d7-4e09-baf1-c3529d823c99 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:41.764774 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4269b3-303e-4178-add4-236ae234e303 service nova] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:41.765004 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e5f3a5-62d7-4e09-baf1-c3529d823c99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:41.766903 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e5f3a5-62d7-4e09-baf1-c3529d823c99 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:41.773546 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 625/1256] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:09:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:09:41.774311 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4269b3-303e-4178-add4-236ae234e303 service nova] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:41.786160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4269b3-303e-4178-add4-236ae234e303 service nova] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:41.794085 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 632/1257] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:09:41 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:09:41.794085 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5f3a5-62d7-4e09-baf1-c3529d823c99 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:41.795257 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e4878c-ab44-49d8-87ec-f80a9e2db434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:41.795868 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4878c-ab44-49d8-87ec-f80a9e2db434 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:41.796126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4878c-ab44-49d8-87ec-f80a9e2db434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:41.796390 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4878c-ab44-49d8-87ec-f80a9e2db434 admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:41.797395 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e4878c-ab44-49d8-87ec-f80a9e2db434 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:41.799335 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e4878c-ab44-49d8-87ec-f80a9e2db434 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:41.806515 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5f3a5-62d7-4e09-baf1-c3529d823c99 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:41.811487 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4430e4-bdfb-480d-a55f-c5ffa1b3c9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:41.812104 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4430e4-bdfb-480d-a55f-c5ffa1b3c9c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:41.812428 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4430e4-bdfb-480d-a55f-c5ffa1b3c9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:41.812730 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4430e4-bdfb-480d-a55f-c5ffa1b3c9c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:41.813374 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4430e4-bdfb-480d-a55f-c5ffa1b3c9c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:41.816708 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4430e4-bdfb-480d-a55f-c5ffa1b3c9c4 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:41.821101 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 626/1258] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:09:41 2022] POST /identity/v3/users => generated 295 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:09:41.821729 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e4878c-ab44-49d8-87ec-f80a9e2db434 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:41.832024 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e4878c-ab44-49d8-87ec-f80a9e2db434 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:41.836924 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07a3f5-d228-4ae8-98e5-fb808da8bb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:41.837807 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07a3f5-d228-4ae8-98e5-fb808da8bb9e admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:41.837807 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07a3f5-d228-4ae8-98e5-fb808da8bb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:41.838096 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07a3f5-d228-4ae8-98e5-fb808da8bb9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:41.839999 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07a3f5-d228-4ae8-98e5-fb808da8bb9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:41.841899 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07a3f5-d228-4ae8-98e5-fb808da8bb9e admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:41.843671 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 633/1259] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:09:41 2022] POST /identity/v3/users => generated 290 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:09:41.844625 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4430e4-bdfb-480d-a55f-c5ffa1b3c9c4 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:41.855151 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4430e4-bdfb-480d-a55f-c5ffa1b3c9c4 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:41.860500 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7f383-2ffd-4206-add1-41fab6791174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:41.861106 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7f383-2ffd-4206-add1-41fab6791174 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:41.861413 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7f383-2ffd-4206-add1-41fab6791174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:41.861687 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7f383-2ffd-4206-add1-41fab6791174 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:41.862347 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf7f383-2ffd-4206-add1-41fab6791174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:41.863990 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf7f383-2ffd-4206-add1-41fab6791174 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:41.873863 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 627/1260] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:09:41 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:09:41.877419 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 634/1261] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:09:41.877419 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7f383-2ffd-4206-add1-41fab6791174 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:41.877649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07a3f5-d228-4ae8-98e5-fb808da8bb9e admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:41.887384 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7f383-2ffd-4206-add1-41fab6791174 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:41.942425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07a3f5-d228-4ae8-98e5-fb808da8bb9e admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:41.947723 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cfe0d-db18-40c9-b4e4-ebfa697a31df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:41.948918 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cfe0d-db18-40c9-b4e4-ebfa697a31df admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:41.949244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cfe0d-db18-40c9-b4e4-ebfa697a31df admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:41.949575 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cfe0d-db18-40c9-b4e4-ebfa697a31df admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:41.950287 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9cfe0d-db18-40c9-b4e4-ebfa697a31df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:41.952383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9cfe0d-db18-40c9-b4e4-ebfa697a31df admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:41.954189 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ed4f9-7aa7-4439-99ca-c0d3b9159526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:41.954887 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ed4f9-7aa7-4439-99ca-c0d3b9159526 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:41.955295 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ed4f9-7aa7-4439-99ca-c0d3b9159526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:41.955706 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ed4f9-7aa7-4439-99ca-c0d3b9159526 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:41.956422 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ed4f9-7aa7-4439-99ca-c0d3b9159526 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:41.958399 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ed4f9-7aa7-4439-99ca-c0d3b9159526 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:41.977628 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 628/1262] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:09:41 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:09:41.978482 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cfe0d-db18-40c9-b4e4-ebfa697a31df admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:41.988980 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cfe0d-db18-40c9-b4e4-ebfa697a31df admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:41.992415 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 635/1263] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:09:41 2022] POST /identity/v3/users => generated 296 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:09:41.992415 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ed4f9-7aa7-4439-99ca-c0d3b9159526 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:41.997019 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f001e65-2eee-4a47-938d-1d1f5cdf95a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:41.997849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f001e65-2eee-4a47-938d-1d1f5cdf95a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:41.997999 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f001e65-2eee-4a47-938d-1d1f5cdf95a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:41.998326 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f001e65-2eee-4a47-938d-1d1f5cdf95a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:41.999304 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f001e65-2eee-4a47-938d-1d1f5cdf95a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:42.001136 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f001e65-2eee-4a47-938d-1d1f5cdf95a8 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:42.001562 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ed4f9-7aa7-4439-99ca-c0d3b9159526 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:42.007032 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c79ba-4da7-47ca-906a-d76d506b39fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:42.008198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c79ba-4da7-47ca-906a-d76d506b39fc admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:42.008661 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c79ba-4da7-47ca-906a-d76d506b39fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:42.009065 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c79ba-4da7-47ca-906a-d76d506b39fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:42.009835 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c79ba-4da7-47ca-906a-d76d506b39fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:42.012064 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c79ba-4da7-47ca-906a-d76d506b39fc admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:42.016494 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 629/1264] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:09:42.016494 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f001e65-2eee-4a47-938d-1d1f5cdf95a8 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:42.026117 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 636/1265] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:09:42.027358 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f001e65-2eee-4a47-938d-1d1f5cdf95a8 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:42.027509 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c79ba-4da7-47ca-906a-d76d506b39fc admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:42.032427 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad77f7-810c-40e6-af98-b0240e6b6543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:42.033130 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad77f7-810c-40e6-af98-b0240e6b6543 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:42.033531 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad77f7-810c-40e6-af98-b0240e6b6543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:42.033900 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad77f7-810c-40e6-af98-b0240e6b6543 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:42.034609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ad77f7-810c-40e6-af98-b0240e6b6543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:42.036288 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ad77f7-810c-40e6-af98-b0240e6b6543 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:42.037051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c79ba-4da7-47ca-906a-d76d506b39fc admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:42.041659 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610fb10e-f87c-4138-b7ad-60c0bf037e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:42.042343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fb10e-f87c-4138-b7ad-60c0bf037e21 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:42.042629 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fb10e-f87c-4138-b7ad-60c0bf037e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:42.042899 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fb10e-f87c-4138-b7ad-60c0bf037e21 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:42.043566 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610fb10e-f87c-4138-b7ad-60c0bf037e21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:42.045639 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610fb10e-f87c-4138-b7ad-60c0bf037e21 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:42.047474 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 630/1266] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:09:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:09:42.048397 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad77f7-810c-40e6-af98-b0240e6b6543 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:42.058683 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad77f7-810c-40e6-af98-b0240e6b6543 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:42.063897 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb38750-7374-47c8-bd0e-28469c877566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:42.064485 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb38750-7374-47c8-bd0e-28469c877566 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:42.064771 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb38750-7374-47c8-bd0e-28469c877566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:42.065044 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb38750-7374-47c8-bd0e-28469c877566 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:42.065680 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb38750-7374-47c8-bd0e-28469c877566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:42.067288 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb38750-7374-47c8-bd0e-28469c877566 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:42.069798 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 637/1267] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:09:42 2022] POST /identity/v3/users => generated 289 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:09:42.070627 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610fb10e-f87c-4138-b7ad-60c0bf037e21 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:42.081868 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 631/1268] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:09:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:09:42.081868 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb38750-7374-47c8-bd0e-28469c877566 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:42.082121 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610fb10e-f87c-4138-b7ad-60c0bf037e21 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:42.086564 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f9b633-4953-452b-8b18-eff0ff9ef378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:42.087254 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9b633-4953-452b-8b18-eff0ff9ef378 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:42.087533 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9b633-4953-452b-8b18-eff0ff9ef378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:42.087806 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9b633-4953-452b-8b18-eff0ff9ef378 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:42.088441 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f9b633-4953-452b-8b18-eff0ff9ef378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:42.090156 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb38750-7374-47c8-bd0e-28469c877566 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:42.090593 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f9b633-4953-452b-8b18-eff0ff9ef378 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:42.094700 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39270c-c1d8-4ea8-83a7-ca32a0d20dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:42.095309 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39270c-c1d8-4ea8-83a7-ca32a0d20dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:42.095592 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39270c-c1d8-4ea8-83a7-ca32a0d20dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:42.095868 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39270c-c1d8-4ea8-83a7-ca32a0d20dd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:42.096675 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39270c-c1d8-4ea8-83a7-ca32a0d20dd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:42.098238 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39270c-c1d8-4ea8-83a7-ca32a0d20dd5 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:42.100647 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 638/1269] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:09:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:09:42.104381 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f9b633-4953-452b-8b18-eff0ff9ef378 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:42.110776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 632/1270] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:09:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:09:42.110776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36ba60-2e4e-44cb-bbd2-dcf67ed86eaf None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:42.110776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36ba60-2e4e-44cb-bbd2-dcf67ed86eaf None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:42.110776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36ba60-2e4e-44cb-bbd2-dcf67ed86eaf None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:42.111274 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 633/1271] 10.222.0.23 () {54 vars in 912 bytes} [Thu Oct 13 13:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:09:42.112118 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d36ba60-2e4e-44cb-bbd2-dcf67ed86eaf None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:42.114749 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f9b633-4953-452b-8b18-eff0ff9ef378 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:42.119662 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a63ba5e-009e-475e-9d96-005b29dc9fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:42.120413 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63ba5e-009e-475e-9d96-005b29dc9fd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:42.120795 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63ba5e-009e-475e-9d96-005b29dc9fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:42.121162 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63ba5e-009e-475e-9d96-005b29dc9fd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:42.121946 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a63ba5e-009e-475e-9d96-005b29dc9fd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:42.123732 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a63ba5e-009e-475e-9d96-005b29dc9fd4 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:42.124310 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d36ba60-2e4e-44cb-bbd2-dcf67ed86eaf None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:42.128857 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9036d-4021-437e-9286-0139e699440f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:42.129850 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9036d-4021-437e-9286-0139e699440f admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:42.129850 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9036d-4021-437e-9286-0139e699440f admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:42.130066 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9036d-4021-437e-9286-0139e699440f admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:42.130561 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c9036d-4021-437e-9286-0139e699440f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:42.132153 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c9036d-4021-437e-9286-0139e699440f admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:42.135091 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 639/1272] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:09:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:09:42.136021 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a63ba5e-009e-475e-9d96-005b29dc9fd4 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:42.145216 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a63ba5e-009e-475e-9d96-005b29dc9fd4 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:42.149280 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 634/1273] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:09:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:09:42.149280 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9036d-4021-437e-9286-0139e699440f admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:42.150175 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72776d-d034-4198-8f08-70a8b3164d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:42.150761 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72776d-d034-4198-8f08-70a8b3164d2b admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:42.151620 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72776d-d034-4198-8f08-70a8b3164d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:42.151926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72776d-d034-4198-8f08-70a8b3164d2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:42.152558 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72776d-d034-4198-8f08-70a8b3164d2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:42.154121 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72776d-d034-4198-8f08-70a8b3164d2b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:42.157447 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9036d-4021-437e-9286-0139e699440f admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:42.161965 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a4684-51bd-45eb-9f01-d3354789fce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:42.162566 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a4684-51bd-45eb-9f01-d3354789fce1 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:42.162849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a4684-51bd-45eb-9f01-d3354789fce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:42.163163 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a4684-51bd-45eb-9f01-d3354789fce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:42.163814 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a4684-51bd-45eb-9f01-d3354789fce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:42.165437 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a4684-51bd-45eb-9f01-d3354789fce1 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:42.176573 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 635/1274] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:09:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:09:42.180091 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a4684-51bd-45eb-9f01-d3354789fce1 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:42.191298 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a4684-51bd-45eb-9f01-d3354789fce1 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:42.196047 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb10749-25da-4cc9-8f8c-4590e9d47b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:42.196755 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb10749-25da-4cc9-8f8c-4590e9d47b3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:42.197010 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb10749-25da-4cc9-8f8c-4590e9d47b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:42.197285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb10749-25da-4cc9-8f8c-4590e9d47b3e admin admin] PATH_INFO: `/v3/projects/7ec0414e1e454d2099bda01207ac1380/users/dc9c24d7fa024b1dafa212cf5bf5c02d/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:42.201530 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:42.215727 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb10749-25da-4cc9-8f8c-4590e9d47b3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ec0414e1e454d2099bda01207ac1380, user_id=dc9c24d7fa024b1dafa212cf5bf5c02d, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:42.218457 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb10749-25da-4cc9-8f8c-4590e9d47b3e admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:42.232901 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 636/1275] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:09:42 2022] PUT /identity/v3/projects/7ec0414e1e454d2099bda01207ac1380/users/dc9c24d7fa024b1dafa212cf5bf5c02d/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:09:42.233766 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb10749-25da-4cc9-8f8c-4590e9d47b3e admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:42.283321 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 640/1276] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:09:42 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:09:42.284209 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72776d-d034-4198-8f08-70a8b3164d2b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:42.293926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72776d-d034-4198-8f08-70a8b3164d2b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:42.305745 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb10749-25da-4cc9-8f8c-4590e9d47b3e admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:42.310849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e5e43-4220-458b-ab7b-c9261cbc3b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:42.311483 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e5e43-4220-458b-ab7b-c9261cbc3b16 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:42.311777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e5e43-4220-458b-ab7b-c9261cbc3b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:42.312029 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e5e43-4220-458b-ab7b-c9261cbc3b16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:42.312663 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1e5e43-4220-458b-ab7b-c9261cbc3b16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:42.314244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1e5e43-4220-458b-ab7b-c9261cbc3b16 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:42.314785 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:42.325309 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 637/1277] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:09:42 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:09:42.326481 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063e555-2a58-46ce-9902-898402e954fe None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:42.326760 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063e555-2a58-46ce-9902-898402e954fe None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:42.327033 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063e555-2a58-46ce-9902-898402e954fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:42.345558 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-0063e555-2a58-46ce-9902-898402e954fe None None] MFA Rules not processed for user `12700d985a724595af630c81e63f86d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:42.346767 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-0063e555-2a58-46ce-9902-898402e954fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:42.353672 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b81d1-07f3-46e8-a4fa-1322fe869b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:42.354302 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b81d1-07f3-46e8-a4fa-1322fe869b7f admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:42.354598 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b81d1-07f3-46e8-a4fa-1322fe869b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:42.354852 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b81d1-07f3-46e8-a4fa-1322fe869b7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:42.355537 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b81d1-07f3-46e8-a4fa-1322fe869b7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:42.357166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b81d1-07f3-46e8-a4fa-1322fe869b7f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:42.366515 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 641/1278] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:09:42 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:09:42.367299 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b81d1-07f3-46e8-a4fa-1322fe869b7f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:42.380681 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b81d1-07f3-46e8-a4fa-1322fe869b7f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:42.385232 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dfe871-ba47-4eb6-8da2-38cc6412245e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:42.386454 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dfe871-ba47-4eb6-8da2-38cc6412245e admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:42.386734 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dfe871-ba47-4eb6-8da2-38cc6412245e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:42.387003 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dfe871-ba47-4eb6-8da2-38cc6412245e admin admin] PATH_INFO: `/v3/projects/0aff939a649349b89a6f166a47698373/users/aa04de0ff2104dc9b39040fb11095d8e/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:42.401295 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dfe871-ba47-4eb6-8da2-38cc6412245e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0aff939a649349b89a6f166a47698373, user_id=aa04de0ff2104dc9b39040fb11095d8e, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:42.404275 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dfe871-ba47-4eb6-8da2-38cc6412245e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:42.413676 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 638/1279] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Oct 13 13:09:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:42.414282 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0063e555-2a58-46ce-9902-898402e954fe None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:42.425900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 642/1280] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:09:42 2022] PUT /identity/v3/projects/0aff939a649349b89a6f166a47698373/users/aa04de0ff2104dc9b39040fb11095d8e/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:09:42.425900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dfe871-ba47-4eb6-8da2-38cc6412245e admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:42.433669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0063e555-2a58-46ce-9902-898402e954fe None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:42.484212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dfe871-ba47-4eb6-8da2-38cc6412245e admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:42.488896 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf1dd7-1db5-4500-9582-25f683604190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:42.489500 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf1dd7-1db5-4500-9582-25f683604190 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:42.489780 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf1dd7-1db5-4500-9582-25f683604190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:42.490054 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf1dd7-1db5-4500-9582-25f683604190 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:42.490692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecf1dd7-1db5-4500-9582-25f683604190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:42.490797 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2aa350-18bd-43f6-ba36-f357e605cfbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:42.491474 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2aa350-18bd-43f6-ba36-f357e605cfbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:42.491880 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2aa350-18bd-43f6-ba36-f357e605cfbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:42.492206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2aa350-18bd-43f6-ba36-f357e605cfbf admin admin] PATH_INFO: `/v3/projects/4af4b9833fd14a1f881610bbc90bbd37/users/b5dbaf53464b44fe8d9e16f5013587b4/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:42.492309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecf1dd7-1db5-4500-9582-25f683604190 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:42.502641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 643/1281] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:09:42 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:09:42.503933 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099db09b-3071-4d02-aebb-1e6b5ebafdc7 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:42.504219 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099db09b-3071-4d02-aebb-1e6b5ebafdc7 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:42.504500 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099db09b-3071-4d02-aebb-1e6b5ebafdc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:42.508788 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2aa350-18bd-43f6-ba36-f357e605cfbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=4af4b9833fd14a1f881610bbc90bbd37, user_id=b5dbaf53464b44fe8d9e16f5013587b4, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:42.512130 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2aa350-18bd-43f6-ba36-f357e605cfbf admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:42.515196 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:42.527608 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 639/1282] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:09:42 2022] PUT /identity/v3/projects/4af4b9833fd14a1f881610bbc90bbd37/users/b5dbaf53464b44fe8d9e16f5013587b4/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:09:42.528389 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2aa350-18bd-43f6-ba36-f357e605cfbf admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:42.533579 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-099db09b-3071-4d02-aebb-1e6b5ebafdc7 None None] MFA Rules not processed for user `dc9c24d7fa024b1dafa212cf5bf5c02d`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:42.534771 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-099db09b-3071-4d02-aebb-1e6b5ebafdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:42.588482 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2aa350-18bd-43f6-ba36-f357e605cfbf admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:42.593069 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf20f8-36e7-4935-87a5-1ee252302d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:42.593725 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf20f8-36e7-4935-87a5-1ee252302d28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:42.594006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf20f8-36e7-4935-87a5-1ee252302d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:42.594285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf20f8-36e7-4935-87a5-1ee252302d28 admin admin] PATH_INFO: `/v3/projects/6027a602300d4d2bbf33ac329b2960f6/users/e3a049e77aba470db21e26d1bc8caa80/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:42.599442 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 644/1283] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:42.603990 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099db09b-3071-4d02-aebb-1e6b5ebafdc7 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:42.611221 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099db09b-3071-4d02-aebb-1e6b5ebafdc7 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:42.613617 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bf20f8-36e7-4935-87a5-1ee252302d28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6027a602300d4d2bbf33ac329b2960f6, user_id=e3a049e77aba470db21e26d1bc8caa80, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:42.615758 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c107fd85-4d76-4057-94e9-a7e1338b9f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:42.616292 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bf20f8-36e7-4935-87a5-1ee252302d28 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:42.616536 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107fd85-4d76-4057-94e9-a7e1338b9f58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:42.616790 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107fd85-4d76-4057-94e9-a7e1338b9f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:42.617057 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107fd85-4d76-4057-94e9-a7e1338b9f58 admin admin] PATH_INFO: `/v3/projects/4b4877965cc4405a9c793fde9f764e4d/users/677a97f17a664152b8b79149e607550b/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:42.634183 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 640/1284] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:09:42 2022] PUT /identity/v3/projects/6027a602300d4d2bbf33ac329b2960f6/users/e3a049e77aba470db21e26d1bc8caa80/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:09:42.635298 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c107fd85-4d76-4057-94e9-a7e1338b9f58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b4877965cc4405a9c793fde9f764e4d, user_id=677a97f17a664152b8b79149e607550b, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:42.638071 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c107fd85-4d76-4057-94e9-a7e1338b9f58 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:42.638567 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf20f8-36e7-4935-87a5-1ee252302d28 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:42.647786 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:42.655099 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 645/1285] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:09:42 2022] PUT /identity/v3/projects/4b4877965cc4405a9c793fde9f764e4d/users/677a97f17a664152b8b79149e607550b/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:09:42.656674 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba02bfa-c35f-420d-8d5b-db3ed7e42a70 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:42.656960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba02bfa-c35f-420d-8d5b-db3ed7e42a70 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:42.657230 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba02bfa-c35f-420d-8d5b-db3ed7e42a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:42.685493 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-dba02bfa-c35f-420d-8d5b-db3ed7e42a70 None None] MFA Rules not processed for user `aa04de0ff2104dc9b39040fb11095d8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:42.686690 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-dba02bfa-c35f-420d-8d5b-db3ed7e42a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:42.707441 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf20f8-36e7-4935-87a5-1ee252302d28 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:42.753856 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 646/1286] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:42.755017 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba02bfa-c35f-420d-8d5b-db3ed7e42a70 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:42.769215 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3527ad-837a-41fc-8619-90396eace532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:42.769881 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3527ad-837a-41fc-8619-90396eace532 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:42.770165 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3527ad-837a-41fc-8619-90396eace532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:42.770438 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3527ad-837a-41fc-8619-90396eace532 admin admin] PATH_INFO: `/v3/projects/5e7312d0ac9248f2bfbece5611913ab2/users/454203f47b434cca952a1cef56de559f/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:42.785019 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3527ad-837a-41fc-8619-90396eace532 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e7312d0ac9248f2bfbece5611913ab2, user_id=454203f47b434cca952a1cef56de559f, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:42.787717 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3527ad-837a-41fc-8619-90396eace532 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:42.802053 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 641/1287] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:09:42 2022] PUT /identity/v3/projects/5e7312d0ac9248f2bfbece5611913ab2/users/454203f47b434cca952a1cef56de559f/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:09:42.803236 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3527ad-837a-41fc-8619-90396eace532 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:42.813078 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:42.818704 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba02bfa-c35f-420d-8d5b-db3ed7e42a70 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:42.865784 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3527ad-837a-41fc-8619-90396eace532 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:42.868500 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91887c77-dd76-4f27-a147-4ec888d55e82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:42.869108 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91887c77-dd76-4f27-a147-4ec888d55e82 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:42.869387 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91887c77-dd76-4f27-a147-4ec888d55e82 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:42.869659 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91887c77-dd76-4f27-a147-4ec888d55e82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:42.871031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce80b59c-50f4-4d5f-9909-d7445aeb53e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:42.871712 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80b59c-50f4-4d5f-9909-d7445aeb53e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:42.872034 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80b59c-50f4-4d5f-9909-d7445aeb53e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:42.872262 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80b59c-50f4-4d5f-9909-d7445aeb53e3 admin admin] PATH_INFO: `/v3/projects/ecd3d5d2a3a54a07ba64f1fea4846c40/users/2e783caddcad473dbdd764919286e244/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:42.887097 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce80b59c-50f4-4d5f-9909-d7445aeb53e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecd3d5d2a3a54a07ba64f1fea4846c40, user_id=2e783caddcad473dbdd764919286e244, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:42.889726 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce80b59c-50f4-4d5f-9909-d7445aeb53e3 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:42.903706 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 642/1288] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:09:42 2022] PUT /identity/v3/projects/ecd3d5d2a3a54a07ba64f1fea4846c40/users/2e783caddcad473dbdd764919286e244/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:09:42.905072 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e0869-831a-4dcf-8a03-c32fd8660091 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:42.905361 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e0869-831a-4dcf-8a03-c32fd8660091 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:42.905663 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e0869-831a-4dcf-8a03-c32fd8660091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:42.927966 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:42.929449 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-bf6e0869-831a-4dcf-8a03-c32fd8660091 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:42.930878 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-bf6e0869-831a-4dcf-8a03-c32fd8660091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:42.943544 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91887c77-dd76-4f27-a147-4ec888d55e82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:42.944981 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91887c77-dd76-4f27-a147-4ec888d55e82 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:42.955781 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 647/1289] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:09:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:09:42.956983 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a441dbf-e644-40a1-a6e8-ba5eac52428d None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:42.957268 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a441dbf-e644-40a1-a6e8-ba5eac52428d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:42.957541 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a441dbf-e644-40a1-a6e8-ba5eac52428d None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:42.958895 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 648/1290] 10.222.0.23 () {54 vars in 961 bytes} [Thu Oct 13 13:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:09:42.959864 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ae31b-084d-4147-84dd-38581c056bca None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:42.960143 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ae31b-084d-4147-84dd-38581c056bca None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:42.960520 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ae31b-084d-4147-84dd-38581c056bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:42.982544 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-216ae31b-084d-4147-84dd-38581c056bca None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:42.983789 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-216ae31b-084d-4147-84dd-38581c056bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:42.997232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 643/1291] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:42.997232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d9c6d-1b59-4811-9533-b48fe1e667f5 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:42.997232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d9c6d-1b59-4811-9533-b48fe1e667f5 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:42.997232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d9c6d-1b59-4811-9533-b48fe1e667f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:43.024727 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-1c6d9c6d-1b59-4811-9533-b48fe1e667f5 None None] MFA Rules not processed for user `677a97f17a664152b8b79149e607550b`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:43.025856 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-1c6d9c6d-1b59-4811-9533-b48fe1e667f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:43.051867 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 649/1292] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:43.053008 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ae31b-084d-4147-84dd-38581c056bca None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:43.101152 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 644/1293] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:43.103048 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760ade3-3b6d-4f3f-9f9d-abfc0b4396cb None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:43.103462 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760ade3-3b6d-4f3f-9f9d-abfc0b4396cb None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:43.103806 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760ade3-3b6d-4f3f-9f9d-abfc0b4396cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:43.114375 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ae31b-084d-4147-84dd-38581c056bca None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:43.118628 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cfad6a-63db-46f4-a57b-9da5ce7b2746 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:43.119264 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cfad6a-63db-46f4-a57b-9da5ce7b2746 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:43.119547 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cfad6a-63db-46f4-a57b-9da5ce7b2746 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:43.119851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cfad6a-63db-46f4-a57b-9da5ce7b2746 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:43.123352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-0760ade3-3b6d-4f3f-9f9d-abfc0b4396cb None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:43.124999 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-0760ade3-3b6d-4f3f-9f9d-abfc0b4396cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:43.134097 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 645/1294] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:43.135399 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc468bf-0fcd-4a2e-b72b-b7342e78975b None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:43.135787 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc468bf-0fcd-4a2e-b72b-b7342e78975b None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:43.136162 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc468bf-0fcd-4a2e-b72b-b7342e78975b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:43.158272 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-8dc468bf-0fcd-4a2e-b72b-b7342e78975b None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:43.159794 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-8dc468bf-0fcd-4a2e-b72b-b7342e78975b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:43.168201 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 646/1295] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:43.169558 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa28daa-ea0c-492a-9d6f-2931e54d4915 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:43.169864 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa28daa-ea0c-492a-9d6f-2931e54d4915 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:43.170121 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa28daa-ea0c-492a-9d6f-2931e54d4915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:43.191109 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-bfa28daa-ea0c-492a-9d6f-2931e54d4915 None None] MFA Rules not processed for user `12700d985a724595af630c81e63f86d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:43.191655 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-bfa28daa-ea0c-492a-9d6f-2931e54d4915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:43.192279 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cfad6a-63db-46f4-a57b-9da5ce7b2746 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:43.194239 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cfad6a-63db-46f4-a57b-9da5ce7b2746 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:43.196275 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:43.206786 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 650/1296] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:09:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:09:43.207952 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cfad6a-63db-46f4-a57b-9da5ce7b2746 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:43.263032 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:43.275503 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 647/1297] 10.222.0.23 () {58 vars in 1058 bytes} [Thu Oct 13 13:09:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:43.277128 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa28daa-ea0c-492a-9d6f-2931e54d4915 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:43.285294 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cfad6a-63db-46f4-a57b-9da5ce7b2746 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:43.286472 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa28daa-ea0c-492a-9d6f-2931e54d4915 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:43.291497 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d3ad4-20bc-4e10-976a-a91e2f8add70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:43.292444 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c8aff-df8d-4e14-9904-df442bde9cb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:43.293338 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d3ad4-20bc-4e10-976a-a91e2f8add70 service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:43.293639 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c8aff-df8d-4e14-9904-df442bde9cb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:43.293816 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d3ad4-20bc-4e10-976a-a91e2f8add70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:43.294025 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d3ad4-20bc-4e10-976a-a91e2f8add70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:43.294229 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c8aff-df8d-4e14-9904-df442bde9cb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:43.294569 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c8aff-df8d-4e14-9904-df442bde9cb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:43.312280 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8d3ad4-20bc-4e10-976a-a91e2f8add70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:43.314365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8d3ad4-20bc-4e10-976a-a91e2f8add70 service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:43.318944 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c8aff-df8d-4e14-9904-df442bde9cb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:43.321605 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c8aff-df8d-4e14-9904-df442bde9cb7 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:43.326530 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 648/1298] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:09:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:09:43.327530 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d3ad4-20bc-4e10-976a-a91e2f8add70 service neutron] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:43.337958 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d3ad4-20bc-4e10-976a-a91e2f8add70 service neutron] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:43.341595 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 651/1299] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:09:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:09:43.342956 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c8aff-df8d-4e14-9904-df442bde9cb7 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:43.343942 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d68442-6760-46da-ad1a-5c56b2939b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:43.344718 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d68442-6760-46da-ad1a-5c56b2939b4e admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:43.344998 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d68442-6760-46da-ad1a-5c56b2939b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:43.345270 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d68442-6760-46da-ad1a-5c56b2939b4e admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:43.345857 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d68442-6760-46da-ad1a-5c56b2939b4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:43.347841 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d68442-6760-46da-ad1a-5c56b2939b4e admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:43.360174 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c8aff-df8d-4e14-9904-df442bde9cb7 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:43.368045 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1423ea5-2aa5-4217-9718-85881f93acf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:43.368776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1423ea5-2aa5-4217-9718-85881f93acf0 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:43.369541 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1423ea5-2aa5-4217-9718-85881f93acf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:43.370052 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1423ea5-2aa5-4217-9718-85881f93acf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:43.387866 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 649/1300] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:09:43 2022] POST /identity/v3/projects => generated 387 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:09:43.389037 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d68442-6760-46da-ad1a-5c56b2939b4e admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:43.410942 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1423ea5-2aa5-4217-9718-85881f93acf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:43.413826 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1423ea5-2aa5-4217-9718-85881f93acf0 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:43.458571 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 652/1301] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:09:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:09:43.460178 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1423ea5-2aa5-4217-9718-85881f93acf0 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:43.465112 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d68442-6760-46da-ad1a-5c56b2939b4e admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:43.469167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c807ab3c-bbf5-4a3c-ac76-3e2d8bf21d7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:43.470232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807ab3c-bbf5-4a3c-ac76-3e2d8bf21d7b service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:43.470644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807ab3c-bbf5-4a3c-ac76-3e2d8bf21d7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:43.471230 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807ab3c-bbf5-4a3c-ac76-3e2d8bf21d7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:43.496939 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1423ea5-2aa5-4217-9718-85881f93acf0 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:43.562648 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c807ab3c-bbf5-4a3c-ac76-3e2d8bf21d7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:43.565285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c807ab3c-bbf5-4a3c-ac76-3e2d8bf21d7b service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:43.581075 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 650/1302] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:09:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:09:43.582021 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c807ab3c-bbf5-4a3c-ac76-3e2d8bf21d7b service neutron] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:43.595970 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c807ab3c-bbf5-4a3c-ac76-3e2d8bf21d7b service neutron] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:43.602536 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb17532-1e83-492f-9ab4-a000ea656290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:43.603288 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb17532-1e83-492f-9ab4-a000ea656290 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:43.603642 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb17532-1e83-492f-9ab4-a000ea656290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:43.604937 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb17532-1e83-492f-9ab4-a000ea656290 admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:43.605640 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb17532-1e83-492f-9ab4-a000ea656290 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:43.608297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb17532-1e83-492f-9ab4-a000ea656290 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:43.634438 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:43.648469 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1e551-dfd2-4b85-a787-2c3e1c270806 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:43.649175 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1e551-dfd2-4b85-a787-2c3e1c270806 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:43.649465 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1e551-dfd2-4b85-a787-2c3e1c270806 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:43.649738 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1e551-dfd2-4b85-a787-2c3e1c270806 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:43.651988 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 651/1303] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:09:43 2022] POST /identity/v3/users => generated 294 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:09:43.652831 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb17532-1e83-492f-9ab4-a000ea656290 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:43.669731 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb17532-1e83-492f-9ab4-a000ea656290 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:43.685663 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda069b-109b-4f99-81fc-553c1be0d066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:43.686134 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda069b-109b-4f99-81fc-553c1be0d066 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:43.686417 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda069b-109b-4f99-81fc-553c1be0d066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:43.686718 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda069b-109b-4f99-81fc-553c1be0d066 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:43.687371 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eda069b-109b-4f99-81fc-553c1be0d066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:43.689145 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eda069b-109b-4f99-81fc-553c1be0d066 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:43.717381 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 652/1304] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:09:43.718300 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda069b-109b-4f99-81fc-553c1be0d066 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:43.740331 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda069b-109b-4f99-81fc-553c1be0d066 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:43.750975 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ebffcb-9800-402b-9aab-c911b8dbbd8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:43.752765 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebffcb-9800-402b-9aab-c911b8dbbd8e admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:43.753155 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebffcb-9800-402b-9aab-c911b8dbbd8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:43.753847 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebffcb-9800-402b-9aab-c911b8dbbd8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:43.754776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ebffcb-9800-402b-9aab-c911b8dbbd8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:43.755277 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1e551-dfd2-4b85-a787-2c3e1c270806 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:43.757822 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1e551-dfd2-4b85-a787-2c3e1c270806 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:43.760758 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ebffcb-9800-402b-9aab-c911b8dbbd8e admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:43.773562 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 653/1305] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:09:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:09:43.803532 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 653/1306] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:09:43.815425 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1e551-dfd2-4b85-a787-2c3e1c270806 service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:43.850928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1e551-dfd2-4b85-a787-2c3e1c270806 service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:43.869759 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39817764-d120-4f8d-88a7-b4f75d94c889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:43.870453 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39817764-d120-4f8d-88a7-b4f75d94c889 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:43.870754 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39817764-d120-4f8d-88a7-b4f75d94c889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:43.871007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39817764-d120-4f8d-88a7-b4f75d94c889 admin admin] PATH_INFO: `/v3/projects/20a7329f9d8d42e29484421023854c1c/users/e6428b84d2f94c18888dbb8008a20a1d/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:43.920109 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39817764-d120-4f8d-88a7-b4f75d94c889 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20a7329f9d8d42e29484421023854c1c, user_id=e6428b84d2f94c18888dbb8008a20a1d, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:43.923260 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39817764-d120-4f8d-88a7-b4f75d94c889 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:43.955370 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 654/1307] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:09:43 2022] PUT /identity/v3/projects/20a7329f9d8d42e29484421023854c1c/users/e6428b84d2f94c18888dbb8008a20a1d/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:09:43.967791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4483a-0c30-4910-90cc-6954e70b5687 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:43.968236 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4483a-0c30-4910-90cc-6954e70b5687 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:43.968679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4483a-0c30-4910-90cc-6954e70b5687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:44.071394 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-1fa4483a-0c30-4910-90cc-6954e70b5687 None None] MFA Rules not processed for user `e6428b84d2f94c18888dbb8008a20a1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:44.080022 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-1fa4483a-0c30-4910-90cc-6954e70b5687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:44.215202 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 654/1308] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:09:44.219020 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39817764-d120-4f8d-88a7-b4f75d94c889 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:44.328443 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:44.355566 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39817764-d120-4f8d-88a7-b4f75d94c889 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:44.362608 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eac782-719e-4d4e-9928-1ee1475ee5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:44.363935 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eac782-719e-4d4e-9928-1ee1475ee5d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:44.364602 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eac782-719e-4d4e-9928-1ee1475ee5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:44.365152 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eac782-719e-4d4e-9928-1ee1475ee5d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:44.366159 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eac782-719e-4d4e-9928-1ee1475ee5d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:44.369552 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eac782-719e-4d4e-9928-1ee1475ee5d0 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:44.426665 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 655/1309] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:09:44 2022] POST /identity/v3/projects => generated 387 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:09:44.430765 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4483a-0c30-4910-90cc-6954e70b5687 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:44.439313 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20397a56-20e6-4348-a07d-86e37158b320 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:44.439621 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20397a56-20e6-4348-a07d-86e37158b320 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:44.439871 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20397a56-20e6-4348-a07d-86e37158b320 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:44.442066 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 656/1310] 10.222.0.23 () {54 vars in 912 bytes} [Thu Oct 13 13:09:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:09:44.447083 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4483a-0c30-4910-90cc-6954e70b5687 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:44.449027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b0fa7-2862-4426-955b-fbb54e89e5b0 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:44.449402 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b0fa7-2862-4426-955b-fbb54e89e5b0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:44.449819 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b0fa7-2862-4426-955b-fbb54e89e5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:44.475581 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-3f7b0fa7-2862-4426-955b-fbb54e89e5b0 None None] MFA Rules not processed for user `12700d985a724595af630c81e63f86d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:44.477140 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3f7b0fa7-2862-4426-955b-fbb54e89e5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:44.528523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fc12c-6bf9-4c9f-86a8-7d026f30a5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:44.529305 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fc12c-6bf9-4c9f-86a8-7d026f30a5c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:44.529788 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fc12c-6bf9-4c9f-86a8-7d026f30a5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:44.530071 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fc12c-6bf9-4c9f-86a8-7d026f30a5c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:44.530843 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174fc12c-6bf9-4c9f-86a8-7d026f30a5c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:44.533895 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174fc12c-6bf9-4c9f-86a8-7d026f30a5c5 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:44.571921 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 655/1311] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:09:44 2022] POST /identity/v3/users => generated 300 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:09:44.577977 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fc12c-6bf9-4c9f-86a8-7d026f30a5c5 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:44.596336 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 657/1312] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Oct 13 13:09:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:09:44.602811 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fc12c-6bf9-4c9f-86a8-7d026f30a5c5 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:44.612263 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b0fa7-2862-4426-955b-fbb54e89e5b0 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:44.624208 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b564a0-e56a-423b-a94f-4cc2718609d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:44.625040 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b564a0-e56a-423b-a94f-4cc2718609d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:44.625319 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b564a0-e56a-423b-a94f-4cc2718609d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:44.627849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b564a0-e56a-423b-a94f-4cc2718609d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:44.628140 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b564a0-e56a-423b-a94f-4cc2718609d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:44.632466 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b564a0-e56a-423b-a94f-4cc2718609d3 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:44.666941 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 656/1313] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:09:44 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:09:44.668345 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b564a0-e56a-423b-a94f-4cc2718609d3 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:44.685328 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b564a0-e56a-423b-a94f-4cc2718609d3 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:44.690169 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213b3f8-d376-4b19-9429-a078144462f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:44.695380 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213b3f8-d376-4b19-9429-a078144462f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:44.695658 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213b3f8-d376-4b19-9429-a078144462f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:44.696179 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213b3f8-d376-4b19-9429-a078144462f0 admin admin] PATH_INFO: `/v3/projects/eaeb18573403411d86a50298a48330d5/users/0f8b329b1b944bf78c97edbb2b7c8ec5/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:44.729714 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b0fa7-2862-4426-955b-fbb54e89e5b0 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:44.738127 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e1b40b-4060-444f-a526-6c4b692c8337 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:44.738918 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1b40b-4060-444f-a526-6c4b692c8337 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:44.739376 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1b40b-4060-444f-a526-6c4b692c8337 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:44.739671 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1b40b-4060-444f-a526-6c4b692c8337 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:44.741683 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213b3f8-d376-4b19-9429-a078144462f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaeb18573403411d86a50298a48330d5, user_id=0f8b329b1b944bf78c97edbb2b7c8ec5, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:44.745619 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213b3f8-d376-4b19-9429-a078144462f0 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:44.755765 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e1b40b-4060-444f-a526-6c4b692c8337 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:44.757339 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e1b40b-4060-444f-a526-6c4b692c8337 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:44.776629 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 658/1314] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:09:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:09:44.780333 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 657/1315] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:09:44 2022] PUT /identity/v3/projects/eaeb18573403411d86a50298a48330d5/users/0f8b329b1b944bf78c97edbb2b7c8ec5/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:09:44.784065 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e1b40b-4060-444f-a526-6c4b692c8337 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:44.798252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e1b40b-4060-444f-a526-6c4b692c8337 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:44.834573 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213b3f8-d376-4b19-9429-a078144462f0 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:44.899503 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:44.933875 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124ddcc-5a00-4299-b4db-b513b4ed12a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:44.934622 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124ddcc-5a00-4299-b4db-b513b4ed12a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:44.934857 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124ddcc-5a00-4299-b4db-b513b4ed12a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:44.935256 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124ddcc-5a00-4299-b4db-b513b4ed12a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:44.936455 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6124ddcc-5a00-4299-b4db-b513b4ed12a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:44.939967 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6124ddcc-5a00-4299-b4db-b513b4ed12a8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:44.957088 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:44.974657 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 659/1316] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:09:44 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:09:44.976885 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124ddcc-5a00-4299-b4db-b513b4ed12a8 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:44.999376 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124ddcc-5a00-4299-b4db-b513b4ed12a8 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:45.008026 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be5117-0760-441d-93cd-0bc4f6c520df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:45.010424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be5117-0760-441d-93cd-0bc4f6c520df admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:45.010840 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be5117-0760-441d-93cd-0bc4f6c520df admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:45.011216 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be5117-0760-441d-93cd-0bc4f6c520df admin admin] PATH_INFO: `/v3/projects/eaeb18573403411d86a50298a48330d5/users/0f8b329b1b944bf78c97edbb2b7c8ec5/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:45.013960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be5117-0760-441d-93cd-0bc4f6c520df admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaeb18573403411d86a50298a48330d5, user_id=0f8b329b1b944bf78c97edbb2b7c8ec5, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:45.015377 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213b3f8-d376-4b19-9429-a078144462f0 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:45.018103 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be5117-0760-441d-93cd-0bc4f6c520df admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:45.033878 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 660/1317] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:09:44 2022] PUT /identity/v3/projects/eaeb18573403411d86a50298a48330d5/users/0f8b329b1b944bf78c97edbb2b7c8ec5/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:09:45.034009 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5006a37f-c6c7-48ca-9b39-d759ccd6b126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:45.034600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5006a37f-c6c7-48ca-9b39-d759ccd6b126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:45.034880 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5006a37f-c6c7-48ca-9b39-d759ccd6b126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:45.035179 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5006a37f-c6c7-48ca-9b39-d759ccd6b126 admin admin] PATH_INFO: `/v3/users/dc9c24d7fa024b1dafa212cf5bf5c02d` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:45.042908 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5006a37f-c6c7-48ca-9b39-d759ccd6b126 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc9c24d7fa024b1dafa212cf5bf5c02d)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:45.045185 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5006a37f-c6c7-48ca-9b39-d759ccd6b126 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:45.061026 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b9304-b068-4adb-a053-821d3991bc76 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:45.061747 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b9304-b068-4adb-a053-821d3991bc76 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:45.062027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b9304-b068-4adb-a053-821d3991bc76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:45.103168 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-130b9304-b068-4adb-a053-821d3991bc76 None None] MFA Rules not processed for user `0f8b329b1b944bf78c97edbb2b7c8ec5`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:45.105926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-130b9304-b068-4adb-a053-821d3991bc76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:45.152692 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-5006a37f-c6c7-48ca-9b39-d759ccd6b126 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc9c24d7fa024b1dafa212cf5bf5c02d'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:45.167665 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-5006a37f-c6c7-48ca-9b39-d759ccd6b126 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc9c24d7fa024b1dafa212cf5bf5c02d'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:45.168980 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-5006a37f-c6c7-48ca-9b39-d759ccd6b126 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc9c24d7fa024b1dafa212cf5bf5c02d'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:45.169912 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-5006a37f-c6c7-48ca-9b39-d759ccd6b126 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc9c24d7fa024b1dafa212cf5bf5c02d'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:45.189623 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-5006a37f-c6c7-48ca-9b39-d759ccd6b126 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc9c24d7fa024b1dafa212cf5bf5c02d'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:45.227256 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 658/1318] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:09:44 2022] DELETE /identity/v3/users/dc9c24d7fa024b1dafa212cf5bf5c02d => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:09:45.232307 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e68f0-e8f1-4bfb-b3bd-2fbc07fca50b None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:45.232638 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e68f0-e8f1-4bfb-b3bd-2fbc07fca50b None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:45.232981 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e68f0-e8f1-4bfb-b3bd-2fbc07fca50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:45.275928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 661/1319] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:45 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:45.283848 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-4f6e68f0-e8f1-4bfb-b3bd-2fbc07fca50b None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:45.290794 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-4f6e68f0-e8f1-4bfb-b3bd-2fbc07fca50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:45.375729 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b9304-b068-4adb-a053-821d3991bc76 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:45.386139 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-130b9304-b068-4adb-a053-821d3991bc76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:45.446596 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 659/1320] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:45.476278 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e68f0-e8f1-4bfb-b3bd-2fbc07fca50b None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:45.477824 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-4f6e68f0-e8f1-4bfb-b3bd-2fbc07fca50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:45.488944 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b9304-b068-4adb-a053-821d3991bc76 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:45.600804 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:45.640862 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc2e8b-9271-4e3e-a8ad-88cbb7501613 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:45.642176 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e68f0-e8f1-4bfb-b3bd-2fbc07fca50b None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:45.643455 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc2e8b-9271-4e3e-a8ad-88cbb7501613 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:45.643734 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc2e8b-9271-4e3e-a8ad-88cbb7501613 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:45.644006 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc2e8b-9271-4e3e-a8ad-88cbb7501613 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:45.652289 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6995d95-4a1e-4d44-a87a-5c358e1e783e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:45.654907 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6995d95-4a1e-4d44-a87a-5c358e1e783e service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:45.655341 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6995d95-4a1e-4d44-a87a-5c358e1e783e service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:45.655780 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6995d95-4a1e-4d44-a87a-5c358e1e783e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:45.677796 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6995d95-4a1e-4d44-a87a-5c358e1e783e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:45.681699 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6995d95-4a1e-4d44-a87a-5c358e1e783e service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:45.707412 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 660/1321] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:09:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:09:45.737629 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:45.788244 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdc2e8b-9271-4e3e-a8ad-88cbb7501613 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:45.792196 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdc2e8b-9271-4e3e-a8ad-88cbb7501613 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:45.827539 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 662/1322] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:09:45 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:09:45.961741 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6995d95-4a1e-4d44-a87a-5c358e1e783e service neutron] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:45.964716 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-b6995d95-4a1e-4d44-a87a-5c358e1e783e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:45.997731 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6995d95-4a1e-4d44-a87a-5c358e1e783e service neutron] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:46.007229 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eecc5b-04c5-4535-aa76-61a68c7a80fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:46.012101 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eecc5b-04c5-4535-aa76-61a68c7a80fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:46.013447 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eecc5b-04c5-4535-aa76-61a68c7a80fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:46.013917 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eecc5b-04c5-4535-aa76-61a68c7a80fa admin admin] PATH_INFO: `/v3/users/8c83931f54cb4d16b530cf7b4b817899` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:46.018093 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eecc5b-04c5-4535-aa76-61a68c7a80fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c83931f54cb4d16b530cf7b4b817899)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:46.023374 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eecc5b-04c5-4535-aa76-61a68c7a80fa admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:46.152611 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-49eecc5b-04c5-4535-aa76-61a68c7a80fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c83931f54cb4d16b530cf7b4b817899'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:46.180978 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-49eecc5b-04c5-4535-aa76-61a68c7a80fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c83931f54cb4d16b530cf7b4b817899'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:46.181900 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-49eecc5b-04c5-4535-aa76-61a68c7a80fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c83931f54cb4d16b530cf7b4b817899'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:46.183441 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-49eecc5b-04c5-4535-aa76-61a68c7a80fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c83931f54cb4d16b530cf7b4b817899'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:46.205005 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-49eecc5b-04c5-4535-aa76-61a68c7a80fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c83931f54cb4d16b530cf7b4b817899'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:46.287151 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 661/1323] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:09:45 2022] DELETE /identity/v3/users/8c83931f54cb4d16b530cf7b4b817899 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:09:46.292886 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685a94c-7799-489f-9874-0e811f8dd055 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:46.293676 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685a94c-7799-489f-9874-0e811f8dd055 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:46.294408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685a94c-7799-489f-9874-0e811f8dd055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:46.330098 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-4685a94c-7799-489f-9874-0e811f8dd055 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:46.332016 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4685a94c-7799-489f-9874-0e811f8dd055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:46.456701 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 663/1324] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:46.487324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eecc5b-04c5-4535-aa76-61a68c7a80fa admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:46.490058 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-49eecc5b-04c5-4535-aa76-61a68c7a80fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:46.645884 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eecc5b-04c5-4535-aa76-61a68c7a80fa admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:46.658870 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced2fb9-d137-4c4b-a7b7-7df6ad9e7f1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:46.659566 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced2fb9-d137-4c4b-a7b7-7df6ad9e7f1b service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:46.659847 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced2fb9-d137-4c4b-a7b7-7df6ad9e7f1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:46.660127 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced2fb9-d137-4c4b-a7b7-7df6ad9e7f1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:46.688214 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aced2fb9-d137-4c4b-a7b7-7df6ad9e7f1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:46.689701 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aced2fb9-d137-4c4b-a7b7-7df6ad9e7f1b service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:46.724504 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 662/1325] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:09:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:09:46.834910 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4685a94c-7799-489f-9874-0e811f8dd055 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:46.836451 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4685a94c-7799-489f-9874-0e811f8dd055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:46.865725 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4685a94c-7799-489f-9874-0e811f8dd055 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:46.874301 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e294879-1c2e-4ca5-8afe-53eebe1a5e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:46.875103 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e294879-1c2e-4ca5-8afe-53eebe1a5e64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:46.875411 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e294879-1c2e-4ca5-8afe-53eebe1a5e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:46.875685 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e294879-1c2e-4ca5-8afe-53eebe1a5e64 admin admin] PATH_INFO: `/v3/projects/7ec0414e1e454d2099bda01207ac1380` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:46.877813 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e294879-1c2e-4ca5-8afe-53eebe1a5e64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ec0414e1e454d2099bda01207ac1380)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:46.881460 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e294879-1c2e-4ca5-8afe-53eebe1a5e64 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:46.987300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2e294879-1c2e-4ca5-8afe-53eebe1a5e64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ec0414e1e454d2099bda01207ac1380', 'request_id': 'req-2e294879-1c2e-4ca5-8afe-53eebe1a5e64'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:47.003966 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2e294879-1c2e-4ca5-8afe-53eebe1a5e64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ec0414e1e454d2099bda01207ac1380', 'request_id': 'req-2e294879-1c2e-4ca5-8afe-53eebe1a5e64'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:47.009577 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced2fb9-d137-4c4b-a7b7-7df6ad9e7f1b service neutron] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:47.011213 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-aced2fb9-d137-4c4b-a7b7-7df6ad9e7f1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:47.023270 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2e294879-1c2e-4ca5-8afe-53eebe1a5e64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ec0414e1e454d2099bda01207ac1380', 'request_id': 'req-2e294879-1c2e-4ca5-8afe-53eebe1a5e64'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:47.050829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2e294879-1c2e-4ca5-8afe-53eebe1a5e64 admin admin] The token cache is being invalidate because project 7ec0414e1e454d2099bda01207ac1380 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:09:47.051229 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2e294879-1c2e-4ca5-8afe-53eebe1a5e64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:47.053287 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2e294879-1c2e-4ca5-8afe-53eebe1a5e64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:47.059508 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 664/1326] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:09:46 2022] DELETE /identity/v3/projects/7ec0414e1e454d2099bda01207ac1380 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:09:47.061084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a203717-104b-4360-822c-e5173ba243ad None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:47.061347 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a203717-104b-4360-822c-e5173ba243ad None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:47.061617 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a203717-104b-4360-822c-e5173ba243ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:47.104809 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6a203717-104b-4360-822c-e5173ba243ad None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:47.106079 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6a203717-104b-4360-822c-e5173ba243ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:47.141587 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced2fb9-d137-4c4b-a7b7-7df6ad9e7f1b service neutron] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:47.162547 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:47.237575 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:47.261296 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b9b03-b790-4f5c-af8a-247561aa9313 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:47.261296 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b9b03-b790-4f5c-af8a-247561aa9313 service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:47.261296 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b9b03-b790-4f5c-af8a-247561aa9313 service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:47.261833 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 665/1327] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:09:47.261929 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b9b03-b790-4f5c-af8a-247561aa9313 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:47.263885 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-8a3b9b03-b790-4f5c-af8a-247561aa9313 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:47.266749 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5d393-a92d-4f67-89fd-3229c1aea7bc None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:47.267009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5d393-a92d-4f67-89fd-3229c1aea7bc None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:47.267465 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5d393-a92d-4f67-89fd-3229c1aea7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:47.339939 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-7aa5d393-a92d-4f67-89fd-3229c1aea7bc None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:47.341169 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7aa5d393-a92d-4f67-89fd-3229c1aea7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:47.361832 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 666/1328] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:47.376375 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5d393-a92d-4f67-89fd-3229c1aea7bc None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:47.403989 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5d393-a92d-4f67-89fd-3229c1aea7bc None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:47.419722 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2463650-81fc-4597-af7c-e0b5f0bfca87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:47.420440 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2463650-81fc-4597-af7c-e0b5f0bfca87 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:47.420722 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2463650-81fc-4597-af7c-e0b5f0bfca87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:47.421234 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2463650-81fc-4597-af7c-e0b5f0bfca87 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:47.421917 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2463650-81fc-4597-af7c-e0b5f0bfca87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:09:47.422220 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2463650-81fc-4597-af7c-e0b5f0bfca87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:47.423877 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2463650-81fc-4597-af7c-e0b5f0bfca87 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:47.433084 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b9b03-b790-4f5c-af8a-247561aa9313 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:47.434622 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b9b03-b790-4f5c-af8a-247561aa9313 service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:47.461037 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 663/1329] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:09:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 13 13:09:47.485393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b9b03-b790-4f5c-af8a-247561aa9313 service cinder] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:47.486855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 667/1330] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:09:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:09:47.502737 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b9b03-b790-4f5c-af8a-247561aa9313 service cinder] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:47.514597 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c54e82-41b8-473b-ab0a-e2b29672601a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:47.515266 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c54e82-41b8-473b-ab0a-e2b29672601a admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:47.515659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c54e82-41b8-473b-ab0a-e2b29672601a admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:47.515953 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c54e82-41b8-473b-ab0a-e2b29672601a admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:47.516667 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c54e82-41b8-473b-ab0a-e2b29672601a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:47.518759 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c54e82-41b8-473b-ab0a-e2b29672601a admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:47.586660 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 664/1331] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:09:47 2022] POST /identity/v3/projects => generated 405 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:09:47.587514 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2463650-81fc-4597-af7c-e0b5f0bfca87 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:47.601403 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2463650-81fc-4597-af7c-e0b5f0bfca87 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:47.742567 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddad58c-7c41-4a9a-97f8-d08388aa54ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:47.743655 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddad58c-7c41-4a9a-97f8-d08388aa54ab admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:47.743943 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddad58c-7c41-4a9a-97f8-d08388aa54ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:47.744241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddad58c-7c41-4a9a-97f8-d08388aa54ab admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:47.744932 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddad58c-7c41-4a9a-97f8-d08388aa54ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:47.746868 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddad58c-7c41-4a9a-97f8-d08388aa54ab admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:47.804848 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 668/1332] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:09:47 2022] POST /identity/v3/users => generated 303 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:09:47.806393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c54e82-41b8-473b-ab0a-e2b29672601a admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:47.820759 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c54e82-41b8-473b-ab0a-e2b29672601a admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:47.837255 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce4a20-3e62-418a-807a-e60281a0870c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:47.838164 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce4a20-3e62-418a-807a-e60281a0870c admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:47.839528 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce4a20-3e62-418a-807a-e60281a0870c admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:47.839820 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce4a20-3e62-418a-807a-e60281a0870c admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:47.841243 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce4a20-3e62-418a-807a-e60281a0870c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:47.846016 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce4a20-3e62-418a-807a-e60281a0870c admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:47.876725 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 665/1333] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:09:47.878191 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddad58c-7c41-4a9a-97f8-d08388aa54ab admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:47.897222 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddad58c-7c41-4a9a-97f8-d08388aa54ab admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:47.913122 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c77ccd-c3c9-4620-a285-7568a589c544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:47.913776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c77ccd-c3c9-4620-a285-7568a589c544 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:47.914064 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c77ccd-c3c9-4620-a285-7568a589c544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:47.915455 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c77ccd-c3c9-4620-a285-7568a589c544 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:47.917339 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c77ccd-c3c9-4620-a285-7568a589c544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:47.921197 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c77ccd-c3c9-4620-a285-7568a589c544 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:47.951097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 669/1334] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:09:47.953318 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce4a20-3e62-418a-807a-e60281a0870c admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:47.975353 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce4a20-3e62-418a-807a-e60281a0870c admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:47.984854 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e795bf-4470-461b-a6ad-802d17806733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:47.985670 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e795bf-4470-461b-a6ad-802d17806733 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:47.986077 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e795bf-4470-461b-a6ad-802d17806733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:47.986461 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e795bf-4470-461b-a6ad-802d17806733 admin admin] PATH_INFO: `/v3/projects/659236f6af5141a2a4a33fb0645a8118/users/f355c5948f244dce979e90836fe0f859/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:48.035160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e795bf-4470-461b-a6ad-802d17806733 admin admin] RBAC: Authorizing `identity:create_grant(project_id=659236f6af5141a2a4a33fb0645a8118, user_id=f355c5948f244dce979e90836fe0f859, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:48.041665 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e795bf-4470-461b-a6ad-802d17806733 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:48.065666 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 666/1335] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:09:47 2022] PUT /identity/v3/projects/659236f6af5141a2a4a33fb0645a8118/users/f355c5948f244dce979e90836fe0f859/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:09:48.082107 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e366e6-84f7-4b65-9e50-1332b13cba09 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:48.082640 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e366e6-84f7-4b65-9e50-1332b13cba09 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:48.083176 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e366e6-84f7-4b65-9e50-1332b13cba09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:48.245959 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-41e366e6-84f7-4b65-9e50-1332b13cba09 None None] MFA Rules not processed for user `f355c5948f244dce979e90836fe0f859`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:48.245959 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-41e366e6-84f7-4b65-9e50-1332b13cba09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:48.408173 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 670/1336] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:48 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:48.412939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e795bf-4470-461b-a6ad-802d17806733 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:48.556850 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e795bf-4470-461b-a6ad-802d17806733 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:48.564507 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f47eb-0bb0-4139-87ec-53ae716eddef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:48.566020 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f47eb-0bb0-4139-87ec-53ae716eddef admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:48.566478 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f47eb-0bb0-4139-87ec-53ae716eddef admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:48.566900 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f47eb-0bb0-4139-87ec-53ae716eddef admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:48.568319 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f47eb-0bb0-4139-87ec-53ae716eddef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:48.573780 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f47eb-0bb0-4139-87ec-53ae716eddef admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:48.596078 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:48.617694 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 667/1337] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:09:48 2022] POST /identity/v3/projects => generated 405 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:09:48.622554 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e366e6-84f7-4b65-9e50-1332b13cba09 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:48.652499 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e366e6-84f7-4b65-9e50-1332b13cba09 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:48.734492 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f47eb-0bb0-4139-87ec-53ae716eddef admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:48.736608 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-a16f47eb-0bb0-4139-87ec-53ae716eddef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:48.761582 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:48.796515 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea828a-23d6-4ed0-9954-7d4f6c5015a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:48.797878 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea828a-23d6-4ed0-9954-7d4f6c5015a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:48.798569 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea828a-23d6-4ed0-9954-7d4f6c5015a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:48.799898 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea828a-23d6-4ed0-9954-7d4f6c5015a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:48.800515 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea828a-23d6-4ed0-9954-7d4f6c5015a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:48.802453 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea828a-23d6-4ed0-9954-7d4f6c5015a6 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:48.844798 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 671/1338] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:09:48 2022] POST /identity/v3/users => generated 309 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:09:48.851030 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea828a-23d6-4ed0-9954-7d4f6c5015a6 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:48.886511 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea828a-23d6-4ed0-9954-7d4f6c5015a6 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:48.897128 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f47eb-0bb0-4139-87ec-53ae716eddef admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:48.905960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c006ad-6077-4751-93c3-154639680b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:48.906597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c006ad-6077-4751-93c3-154639680b8e admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:48.906882 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c006ad-6077-4751-93c3-154639680b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:48.907185 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c006ad-6077-4751-93c3-154639680b8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:48.907848 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c006ad-6077-4751-93c3-154639680b8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:48.914220 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c006ad-6077-4751-93c3-154639680b8e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:48.934639 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209007d0-3eaa-4b59-8072-dc5aeb6c9a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:48.935341 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209007d0-3eaa-4b59-8072-dc5aeb6c9a82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:48.935630 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209007d0-3eaa-4b59-8072-dc5aeb6c9a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:48.935882 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209007d0-3eaa-4b59-8072-dc5aeb6c9a82 admin admin] PATH_INFO: `/v3/projects/fcc66b8f231d4b47be0c6c6b4b52b389` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:48.939810 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209007d0-3eaa-4b59-8072-dc5aeb6c9a82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcc66b8f231d4b47be0c6c6b4b52b389)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:48.940855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: WARNING py.warnings [None req-209007d0-3eaa-4b59-8072-dc5aeb6c9a82 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 13 13:09:48.940855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: warnings.warn(msg) Oct 13 13:09:48.940855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: Oct 13 13:09:48.941440 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 672/1339] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:09:48 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:09:48.942049 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209007d0-3eaa-4b59-8072-dc5aeb6c9a82 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:48.945565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c006ad-6077-4751-93c3-154639680b8e admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:48.958517 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c006ad-6077-4751-93c3-154639680b8e admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:48.973169 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a06e2c7-ef4b-412a-b2d8-5cc429cb0d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:48.973837 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06e2c7-ef4b-412a-b2d8-5cc429cb0d27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:48.974120 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06e2c7-ef4b-412a-b2d8-5cc429cb0d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:48.974396 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06e2c7-ef4b-412a-b2d8-5cc429cb0d27 admin admin] PATH_INFO: `/v3/projects/da0a6f16bd914dc888bec03ce1276df8/users/d787f75042d644a3be31a4e4a54b277e/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:49.038531 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a06e2c7-ef4b-412a-b2d8-5cc429cb0d27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da0a6f16bd914dc888bec03ce1276df8, user_id=d787f75042d644a3be31a4e4a54b277e, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:49.042195 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a06e2c7-ef4b-412a-b2d8-5cc429cb0d27 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:49.077755 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 673/1340] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:09:48 2022] PUT /identity/v3/projects/da0a6f16bd914dc888bec03ce1276df8/users/d787f75042d644a3be31a4e4a54b277e/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:09:49.080165 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-209007d0-3eaa-4b59-8072-dc5aeb6c9a82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcc66b8f231d4b47be0c6c6b4b52b389', 'request_id': 'req-209007d0-3eaa-4b59-8072-dc5aeb6c9a82'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:49.081984 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a06e2c7-ef4b-412a-b2d8-5cc429cb0d27 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:49.098848 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-209007d0-3eaa-4b59-8072-dc5aeb6c9a82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcc66b8f231d4b47be0c6c6b4b52b389', 'request_id': 'req-209007d0-3eaa-4b59-8072-dc5aeb6c9a82'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:49.138135 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-209007d0-3eaa-4b59-8072-dc5aeb6c9a82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcc66b8f231d4b47be0c6c6b4b52b389', 'request_id': 'req-209007d0-3eaa-4b59-8072-dc5aeb6c9a82'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:49.160066 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-209007d0-3eaa-4b59-8072-dc5aeb6c9a82 admin admin] The token cache is being invalidate because project fcc66b8f231d4b47be0c6c6b4b52b389 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:09:49.160355 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-209007d0-3eaa-4b59-8072-dc5aeb6c9a82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:49.161074 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-209007d0-3eaa-4b59-8072-dc5aeb6c9a82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:49.167398 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 668/1341] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:09:48 2022] DELETE /identity/v3/projects/fcc66b8f231d4b47be0c6c6b4b52b389 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:09:49.174271 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa74a0a-c8a6-4922-a77e-1d2bb0744143 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:49.174662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa74a0a-c8a6-4922-a77e-1d2bb0744143 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:49.174856 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa74a0a-c8a6-4922-a77e-1d2bb0744143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:49.238005 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-0aa74a0a-c8a6-4922-a77e-1d2bb0744143 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:49.241034 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-0aa74a0a-c8a6-4922-a77e-1d2bb0744143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:49.249631 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a06e2c7-ef4b-412a-b2d8-5cc429cb0d27 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:49.270419 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3ba61-4cfa-46ff-a16b-0a7f95cc0e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:49.272514 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3ba61-4cfa-46ff-a16b-0a7f95cc0e93 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:49.272802 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 669/1342] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:49.273256 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3ba61-4cfa-46ff-a16b-0a7f95cc0e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:49.273539 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3ba61-4cfa-46ff-a16b-0a7f95cc0e93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:49.274204 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f3ba61-4cfa-46ff-a16b-0a7f95cc0e93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:49.276706 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f3ba61-4cfa-46ff-a16b-0a7f95cc0e93 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:49.280062 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbe682-786b-4e89-b3b9-798cc9866791 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:49.280352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbe682-786b-4e89-b3b9-798cc9866791 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:49.280626 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbe682-786b-4e89-b3b9-798cc9866791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:49.314081 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 674/1343] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:09:49.315697 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3ba61-4cfa-46ff-a16b-0a7f95cc0e93 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:49.343475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c4f3ba61-4cfa-46ff-a16b-0a7f95cc0e93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:49.358755 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-a1dbe682-786b-4e89-b3b9-798cc9866791 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:49.361913 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-a1dbe682-786b-4e89-b3b9-798cc9866791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:49.379467 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3ba61-4cfa-46ff-a16b-0a7f95cc0e93 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:49.388071 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 670/1344] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:49.390442 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dbe682-786b-4e89-b3b9-798cc9866791 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:49.399938 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e12866b-997e-4d7f-812d-bb2996dcb2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:49.404778 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12866b-997e-4d7f-812d-bb2996dcb2ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:49.405325 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12866b-997e-4d7f-812d-bb2996dcb2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:49.407198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12866b-997e-4d7f-812d-bb2996dcb2ca admin admin] PATH_INFO: `/v3/projects/da0a6f16bd914dc888bec03ce1276df8/users/d787f75042d644a3be31a4e4a54b277e/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:49.417845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e12866b-997e-4d7f-812d-bb2996dcb2ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=da0a6f16bd914dc888bec03ce1276df8, user_id=d787f75042d644a3be31a4e4a54b277e, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:49.429422 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e12866b-997e-4d7f-812d-bb2996dcb2ca admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:49.433058 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dbe682-786b-4e89-b3b9-798cc9866791 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:49.471146 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f7bea-1953-4531-84e9-b0203ab0fc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:49.471695 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f7bea-1953-4531-84e9-b0203ab0fc0b admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:49.471809 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f7bea-1953-4531-84e9-b0203ab0fc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:49.472100 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f7bea-1953-4531-84e9-b0203ab0fc0b admin admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:49.472821 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214f7bea-1953-4531-84e9-b0203ab0fc0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:09:49.473130 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214f7bea-1953-4531-84e9-b0203ab0fc0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:49.474754 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214f7bea-1953-4531-84e9-b0203ab0fc0b admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:49.477001 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 675/1345] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:09:49 2022] PUT /identity/v3/projects/da0a6f16bd914dc888bec03ce1276df8/users/d787f75042d644a3be31a4e4a54b277e/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:09:49.489562 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd220ec4-cf25-470a-a779-bc230e8797a1 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:49.489863 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd220ec4-cf25-470a-a779-bc230e8797a1 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:49.490142 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd220ec4-cf25-470a-a779-bc230e8797a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:49.532268 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 671/1346] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:09:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:09:49.535149 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f7bea-1953-4531-84e9-b0203ab0fc0b admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:49.555399 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f7bea-1953-4531-84e9-b0203ab0fc0b admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:49.600132 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-fd220ec4-cf25-470a-a779-bc230e8797a1 None None] MFA Rules not processed for user `d787f75042d644a3be31a4e4a54b277e`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:49.601363 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fd220ec4-cf25-470a-a779-bc230e8797a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:49.762102 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4382f0-6b7a-41b6-9749-1854742a9e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:49.762827 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4382f0-6b7a-41b6-9749-1854742a9e55 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:49.763175 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4382f0-6b7a-41b6-9749-1854742a9e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:49.763454 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4382f0-6b7a-41b6-9749-1854742a9e55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:49.764101 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4382f0-6b7a-41b6-9749-1854742a9e55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:49.767404 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4382f0-6b7a-41b6-9749-1854742a9e55 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:49.784979 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:49.801504 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 676/1347] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:49 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:09:49.812473 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd220ec4-cf25-470a-a779-bc230e8797a1 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:49.813974 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fd220ec4-cf25-470a-a779-bc230e8797a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:49.834812 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:49.845787 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 672/1348] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:09:49 2022] POST /identity/v3/projects => generated 365 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:09:49.846940 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4382f0-6b7a-41b6-9749-1854742a9e55 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:49.965258 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd220ec4-cf25-470a-a779-bc230e8797a1 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:50.023177 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4382f0-6b7a-41b6-9749-1854742a9e55 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:50.037544 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81775cc-99e4-4d00-bcaa-db21978bf64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:50.038734 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81775cc-99e4-4d00-bcaa-db21978bf64b admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:50.039019 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81775cc-99e4-4d00-bcaa-db21978bf64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:50.039322 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81775cc-99e4-4d00-bcaa-db21978bf64b admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:50.039976 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81775cc-99e4-4d00-bcaa-db21978bf64b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:50.046302 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81775cc-99e4-4d00-bcaa-db21978bf64b admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:50.086849 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aad78d-e009-4dd4-9a99-49ab1ccb0679 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:50.087566 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aad78d-e009-4dd4-9a99-49ab1ccb0679 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:50.087866 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aad78d-e009-4dd4-9a99-49ab1ccb0679 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:50.088734 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aad78d-e009-4dd4-9a99-49ab1ccb0679 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:50.095509 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 673/1349] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:09:49 2022] POST /identity/v3/users => generated 283 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:09:50.096493 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81775cc-99e4-4d00-bcaa-db21978bf64b admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:50.122057 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81775cc-99e4-4d00-bcaa-db21978bf64b admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:50.174379 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb14156-6327-4088-8248-b846c32961c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:50.180206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb14156-6327-4088-8248-b846c32961c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:50.182835 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb14156-6327-4088-8248-b846c32961c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:50.185659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb14156-6327-4088-8248-b846c32961c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:50.186828 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb14156-6327-4088-8248-b846c32961c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:50.191412 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb14156-6327-4088-8248-b846c32961c4 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:50.217181 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 674/1350] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:09:50.224419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb14156-6327-4088-8248-b846c32961c4 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:50.255695 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb14156-6327-4088-8248-b846c32961c4 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:50.267545 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aad78d-e009-4dd4-9a99-49ab1ccb0679 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:50.269399 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00168881-16a7-4bab-b8b3-a45a859e2a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:50.270041 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00168881-16a7-4bab-b8b3-a45a859e2a90 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:50.270317 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00168881-16a7-4bab-b8b3-a45a859e2a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:50.270591 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00168881-16a7-4bab-b8b3-a45a859e2a90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:50.272475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aad78d-e009-4dd4-9a99-49ab1ccb0679 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:50.276543 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00168881-16a7-4bab-b8b3-a45a859e2a90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:50.278939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00168881-16a7-4bab-b8b3-a45a859e2a90 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:50.304443 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 677/1351] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:09:49 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 13 13:09:50.324136 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 675/1352] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:09:50.325604 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aad78d-e009-4dd4-9a99-49ab1ccb0679 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:50.346084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aad78d-e009-4dd4-9a99-49ab1ccb0679 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:50.351502 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ef6d6-e702-49de-a3f1-d8c1ae87a259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:50.352247 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ef6d6-e702-49de-a3f1-d8c1ae87a259 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:50.352532 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ef6d6-e702-49de-a3f1-d8c1ae87a259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:50.352844 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ef6d6-e702-49de-a3f1-d8c1ae87a259 admin admin] PATH_INFO: `/v3/projects/157902d713ed4996a65d061aba4e4312/users/da29e093d8bd48f781d5f7874869c572/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:50.385742 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ef6d6-e702-49de-a3f1-d8c1ae87a259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=157902d713ed4996a65d061aba4e4312, user_id=da29e093d8bd48f781d5f7874869c572, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:50.389350 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ef6d6-e702-49de-a3f1-d8c1ae87a259 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:50.423122 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 678/1353] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:09:50 2022] PUT /identity/v3/projects/157902d713ed4996a65d061aba4e4312/users/da29e093d8bd48f781d5f7874869c572/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:09:50.436614 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d027d-9bbb-4a3d-8e81-6b34a70d5223 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:50.436907 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d027d-9bbb-4a3d-8e81-6b34a70d5223 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:50.439149 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d027d-9bbb-4a3d-8e81-6b34a70d5223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:50.525351 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-c23d027d-9bbb-4a3d-8e81-6b34a70d5223 None None] MFA Rules not processed for user `da29e093d8bd48f781d5f7874869c572`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:50.527179 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-c23d027d-9bbb-4a3d-8e81-6b34a70d5223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:50.584484 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ef6d6-e702-49de-a3f1-d8c1ae87a259 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:50.726694 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:50.758082 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 676/1354] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:50.758939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23d027d-9bbb-4a3d-8e81-6b34a70d5223 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:50.771216 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ef6d6-e702-49de-a3f1-d8c1ae87a259 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:50.781555 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23d027d-9bbb-4a3d-8e81-6b34a70d5223 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:50.783824 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a0dd3-4a1a-48cc-983a-9db4ef9e71e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:50.785380 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a0dd3-4a1a-48cc-983a-9db4ef9e71e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:50.785380 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a0dd3-4a1a-48cc-983a-9db4ef9e71e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:50.785380 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a0dd3-4a1a-48cc-983a-9db4ef9e71e2 admin admin] PATH_INFO: `/v3/users/f355c5948f244dce979e90836fe0f859` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:50.788602 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a0dd3-4a1a-48cc-983a-9db4ef9e71e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f355c5948f244dce979e90836fe0f859)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:50.790659 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: WARNING py.warnings [None req-3e1a0dd3-4a1a-48cc-983a-9db4ef9e71e2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 13 13:09:50.790659 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: warnings.warn(msg) Oct 13 13:09:50.790659 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: Oct 13 13:09:50.793419 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a0dd3-4a1a-48cc-983a-9db4ef9e71e2 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:50.794868 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1b307-36fa-4f3b-884b-c7c1be346bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:50.798013 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1b307-36fa-4f3b-884b-c7c1be346bde admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:50.799000 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1b307-36fa-4f3b-884b-c7c1be346bde admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:50.800271 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1b307-36fa-4f3b-884b-c7c1be346bde admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:50.806525 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee1b307-36fa-4f3b-884b-c7c1be346bde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:50.808046 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee1b307-36fa-4f3b-884b-c7c1be346bde admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:50.870230 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 677/1355] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:09:50 2022] POST /identity/v3/projects => generated 365 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:09:50.872079 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1b307-36fa-4f3b-884b-c7c1be346bde admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:50.937330 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:50.955101 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3e1a0dd3-4a1a-48cc-983a-9db4ef9e71e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f355c5948f244dce979e90836fe0f859'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:50.995612 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3e1a0dd3-4a1a-48cc-983a-9db4ef9e71e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f355c5948f244dce979e90836fe0f859'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:51.000290 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3e1a0dd3-4a1a-48cc-983a-9db4ef9e71e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f355c5948f244dce979e90836fe0f859'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:51.022390 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3e1a0dd3-4a1a-48cc-983a-9db4ef9e71e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f355c5948f244dce979e90836fe0f859'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:51.113021 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1b307-36fa-4f3b-884b-c7c1be346bde admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:51.114857 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3e1a0dd3-4a1a-48cc-983a-9db4ef9e71e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f355c5948f244dce979e90836fe0f859'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:51.123702 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee5447-7364-4f43-8613-37d206945b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:51.124448 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee5447-7364-4f43-8613-37d206945b87 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:51.124977 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee5447-7364-4f43-8613-37d206945b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:51.125361 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee5447-7364-4f43-8613-37d206945b87 admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:51.126097 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee5447-7364-4f43-8613-37d206945b87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:51.132116 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 679/1356] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:09:50 2022] DELETE /identity/v3/users/f355c5948f244dce979e90836fe0f859 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:09:51.133050 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee5447-7364-4f43-8613-37d206945b87 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:51.136771 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031185bb-f4d0-4df3-9a5a-0362e06f5215 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:51.137169 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031185bb-f4d0-4df3-9a5a-0362e06f5215 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:51.137551 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031185bb-f4d0-4df3-9a5a-0362e06f5215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:51.183516 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 678/1357] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:09:50 2022] POST /identity/v3/users => generated 289 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:09:51.185123 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee5447-7364-4f43-8613-37d206945b87 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:51.190621 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-96ee5447-7364-4f43-8613-37d206945b87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:51.191836 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-031185bb-f4d0-4df3-9a5a-0362e06f5215 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:51.193173 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-031185bb-f4d0-4df3-9a5a-0362e06f5215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:51.239204 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee5447-7364-4f43-8613-37d206945b87 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:51.360149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 680/1358] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:51.390709 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031185bb-f4d0-4df3-9a5a-0362e06f5215 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:51.392457 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-031185bb-f4d0-4df3-9a5a-0362e06f5215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:51.446419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f800b7-db3c-4688-8f03-9ad8d80f9ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:51.447197 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f800b7-db3c-4688-8f03-9ad8d80f9ad9 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:51.449154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f800b7-db3c-4688-8f03-9ad8d80f9ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:51.449543 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f800b7-db3c-4688-8f03-9ad8d80f9ad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:51.450434 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f800b7-db3c-4688-8f03-9ad8d80f9ad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:51.465284 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f800b7-db3c-4688-8f03-9ad8d80f9ad9 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:51.508524 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 679/1359] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:09:51.514273 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f800b7-db3c-4688-8f03-9ad8d80f9ad9 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:51.528833 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f800b7-db3c-4688-8f03-9ad8d80f9ad9 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:51.534935 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c7c1e-b9c5-4097-bdfc-9cd8167ad110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:51.535558 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c7c1e-b9c5-4097-bdfc-9cd8167ad110 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:51.535912 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c7c1e-b9c5-4097-bdfc-9cd8167ad110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:51.536107 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c7c1e-b9c5-4097-bdfc-9cd8167ad110 admin admin] PATH_INFO: `/v3/projects/dae34432cef2481e861e1ea3ce6dad77/users/8f001ff5529f4598b5ebd9502ab25176/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:51.558517 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031185bb-f4d0-4df3-9a5a-0362e06f5215 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:51.566444 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4c7c1e-b9c5-4097-bdfc-9cd8167ad110 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dae34432cef2481e861e1ea3ce6dad77, user_id=8f001ff5529f4598b5ebd9502ab25176, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:51.571624 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4c7c1e-b9c5-4097-bdfc-9cd8167ad110 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:51.573127 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb84567-3149-428a-8217-6b0449410d76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:51.574262 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb84567-3149-428a-8217-6b0449410d76 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:51.574654 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb84567-3149-428a-8217-6b0449410d76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:51.575100 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb84567-3149-428a-8217-6b0449410d76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:51.600541 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 680/1360] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:09:51 2022] PUT /identity/v3/projects/dae34432cef2481e861e1ea3ce6dad77/users/8f001ff5529f4598b5ebd9502ab25176/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:09:51.604004 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb84567-3149-428a-8217-6b0449410d76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:51.604205 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c7c1e-b9c5-4097-bdfc-9cd8167ad110 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:51.606016 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb84567-3149-428a-8217-6b0449410d76 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:51.631969 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 681/1361] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:09:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 13 13:09:51.739468 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c7c1e-b9c5-4097-bdfc-9cd8167ad110 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:51.748285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dbecc9-f5c2-4ad3-86c7-6c5d109e7411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:51.749133 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbecc9-f5c2-4ad3-86c7-6c5d109e7411 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:51.749514 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbecc9-f5c2-4ad3-86c7-6c5d109e7411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:51.749891 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbecc9-f5c2-4ad3-86c7-6c5d109e7411 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:51.750647 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dbecc9-f5c2-4ad3-86c7-6c5d109e7411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:51.752495 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dbecc9-f5c2-4ad3-86c7-6c5d109e7411 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:51.775360 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb84567-3149-428a-8217-6b0449410d76 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:51.790184 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 681/1362] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:09:51.790729 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb84567-3149-428a-8217-6b0449410d76 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:51.798684 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e2c08e-2b15-42eb-826f-6721c9bf486a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:51.811265 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e2c08e-2b15-42eb-826f-6721c9bf486a admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:51.811558 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e2c08e-2b15-42eb-826f-6721c9bf486a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:51.813093 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e2c08e-2b15-42eb-826f-6721c9bf486a admin admin] PATH_INFO: `/v3/projects/dae34432cef2481e861e1ea3ce6dad77/users/8f001ff5529f4598b5ebd9502ab25176/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:51.817928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e2c08e-2b15-42eb-826f-6721c9bf486a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dae34432cef2481e861e1ea3ce6dad77, user_id=8f001ff5529f4598b5ebd9502ab25176, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:51.827536 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e2c08e-2b15-42eb-826f-6721c9bf486a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:51.854332 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 682/1363] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:09:51 2022] PUT /identity/v3/projects/dae34432cef2481e861e1ea3ce6dad77/users/8f001ff5529f4598b5ebd9502ab25176/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:09:51.865124 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dab498-ea57-457e-8119-5bf692e83130 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:51.865462 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dab498-ea57-457e-8119-5bf692e83130 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:51.865739 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dab498-ea57-457e-8119-5bf692e83130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:51.934678 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-39dab498-ea57-457e-8119-5bf692e83130 None None] MFA Rules not processed for user `8f001ff5529f4598b5ebd9502ab25176`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:51.937129 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-39dab498-ea57-457e-8119-5bf692e83130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:51.948642 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:52.092928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 682/1364] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:51 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:52.113997 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e2c08e-2b15-42eb-826f-6721c9bf486a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:52.117674 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-23e2c08e-2b15-42eb-826f-6721c9bf486a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:52.128055 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:52.304403 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e2c08e-2b15-42eb-826f-6721c9bf486a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:52.309148 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba280d-f43f-41cd-985b-8198d03a76fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:52.309910 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba280d-f43f-41cd-985b-8198d03a76fc service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:52.310193 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba280d-f43f-41cd-985b-8198d03a76fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:52.310464 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba280d-f43f-41cd-985b-8198d03a76fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:52.333656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba280d-f43f-41cd-985b-8198d03a76fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:52.334957 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba280d-f43f-41cd-985b-8198d03a76fc service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:52.349527 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 683/1365] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:09:52 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:09:52.629726 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dab498-ea57-457e-8119-5bf692e83130 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:52.631594 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-39dab498-ea57-457e-8119-5bf692e83130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:52.665855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dab498-ea57-457e-8119-5bf692e83130 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:52.715838 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01acfc-487b-4e01-ae58-de6eb34cd7b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:52.716620 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01acfc-487b-4e01-ae58-de6eb34cd7b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:52.717249 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01acfc-487b-4e01-ae58-de6eb34cd7b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:52.720856 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01acfc-487b-4e01-ae58-de6eb34cd7b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:52.727093 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-5f01acfc-487b-4e01-ae58-de6eb34cd7b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:52.806594 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba280d-f43f-41cd-985b-8198d03a76fc service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:52.808477 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4eba280d-f43f-41cd-985b-8198d03a76fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:52.919360 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba280d-f43f-41cd-985b-8198d03a76fc service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:52.924614 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4fb0f-55cd-49b3-8967-9cc4ed0e82bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:52.925266 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4fb0f-55cd-49b3-8967-9cc4ed0e82bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:52.925639 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4fb0f-55cd-49b3-8967-9cc4ed0e82bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:52.925847 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4fb0f-55cd-49b3-8967-9cc4ed0e82bb admin admin] PATH_INFO: `/v3/projects/659236f6af5141a2a4a33fb0645a8118` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:52.927792 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b4fb0f-55cd-49b3-8967-9cc4ed0e82bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=659236f6af5141a2a4a33fb0645a8118)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:52.930529 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b4fb0f-55cd-49b3-8967-9cc4ed0e82bb admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:52.952087 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f01acfc-487b-4e01-ae58-de6eb34cd7b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:52.955906 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f01acfc-487b-4e01-ae58-de6eb34cd7b0 service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:52.967868 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 683/1366] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:09:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:09:53.054917 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-58b4fb0f-55cd-49b3-8967-9cc4ed0e82bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '659236f6af5141a2a4a33fb0645a8118', 'request_id': 'req-58b4fb0f-55cd-49b3-8967-9cc4ed0e82bb'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:53.070359 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-58b4fb0f-55cd-49b3-8967-9cc4ed0e82bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '659236f6af5141a2a4a33fb0645a8118', 'request_id': 'req-58b4fb0f-55cd-49b3-8967-9cc4ed0e82bb'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:53.094126 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-58b4fb0f-55cd-49b3-8967-9cc4ed0e82bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '659236f6af5141a2a4a33fb0645a8118', 'request_id': 'req-58b4fb0f-55cd-49b3-8967-9cc4ed0e82bb'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:53.141398 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-58b4fb0f-55cd-49b3-8967-9cc4ed0e82bb admin admin] The token cache is being invalidate because project 659236f6af5141a2a4a33fb0645a8118 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:09:53.141623 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-58b4fb0f-55cd-49b3-8967-9cc4ed0e82bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:53.156407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-58b4fb0f-55cd-49b3-8967-9cc4ed0e82bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:53.158249 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 684/1367] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:09:52 2022] DELETE /identity/v3/projects/659236f6af5141a2a4a33fb0645a8118 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:09:53.162352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01acfc-487b-4e01-ae58-de6eb34cd7b0 service cinder] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:53.178969 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01acfc-487b-4e01-ae58-de6eb34cd7b0 service cinder] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:53.325284 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a418e26-191b-49a0-9c7f-4513cc251044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:53.327228 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a418e26-191b-49a0-9c7f-4513cc251044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:53.327577 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a418e26-191b-49a0-9c7f-4513cc251044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:53.328526 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a418e26-191b-49a0-9c7f-4513cc251044 admin admin] PATH_INFO: `/v3/users/d787f75042d644a3be31a4e4a54b277e` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:53.332101 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a418e26-191b-49a0-9c7f-4513cc251044 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d787f75042d644a3be31a4e4a54b277e)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:53.337453 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a418e26-191b-49a0-9c7f-4513cc251044 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:53.372631 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4fb0f-55cd-49b3-8967-9cc4ed0e82bb admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:53.374541 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-58b4fb0f-55cd-49b3-8967-9cc4ed0e82bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:53.527904 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:53.542095 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:53.549394 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-8a418e26-191b-49a0-9c7f-4513cc251044 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd787f75042d644a3be31a4e4a54b277e'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:53.569331 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-8a418e26-191b-49a0-9c7f-4513cc251044 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd787f75042d644a3be31a4e4a54b277e'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:53.570848 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-8a418e26-191b-49a0-9c7f-4513cc251044 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd787f75042d644a3be31a4e4a54b277e'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:53.572064 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-8a418e26-191b-49a0-9c7f-4513cc251044 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd787f75042d644a3be31a4e4a54b277e'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:53.589524 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-8a418e26-191b-49a0-9c7f-4513cc251044 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd787f75042d644a3be31a4e4a54b277e'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:53.633811 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b4fb0f-55cd-49b3-8967-9cc4ed0e82bb admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:53.647998 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 684/1368] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:09:53 2022] DELETE /identity/v3/users/d787f75042d644a3be31a4e4a54b277e => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:09:53.648205 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba453bb7-5d7a-43b4-ae57-6ddce7c5eed5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:53.648205 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba453bb7-5d7a-43b4-ae57-6ddce7c5eed5 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:53.648205 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba453bb7-5d7a-43b4-ae57-6ddce7c5eed5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:53.648205 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba453bb7-5d7a-43b4-ae57-6ddce7c5eed5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:53.653442 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ba453bb7-5d7a-43b4-ae57-6ddce7c5eed5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:53.868098 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba453bb7-5d7a-43b4-ae57-6ddce7c5eed5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:53.870165 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba453bb7-5d7a-43b4-ae57-6ddce7c5eed5 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:53.889783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 685/1369] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:09:53 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 13 13:09:55.239799 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a418e26-191b-49a0-9c7f-4513cc251044 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:55.242037 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-8a418e26-191b-49a0-9c7f-4513cc251044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:55.450908 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a418e26-191b-49a0-9c7f-4513cc251044 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:55.459986 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f39a0e-f4ed-4cd1-84a4-7ff272f21332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:55.462138 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f39a0e-f4ed-4cd1-84a4-7ff272f21332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:55.462570 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f39a0e-f4ed-4cd1-84a4-7ff272f21332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:55.462879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f39a0e-f4ed-4cd1-84a4-7ff272f21332 admin admin] PATH_INFO: `/v3/projects/da0a6f16bd914dc888bec03ce1276df8` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:55.474659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f39a0e-f4ed-4cd1-84a4-7ff272f21332 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da0a6f16bd914dc888bec03ce1276df8)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:55.483323 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f39a0e-f4ed-4cd1-84a4-7ff272f21332 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:55.614671 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-28f39a0e-f4ed-4cd1-84a4-7ff272f21332 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da0a6f16bd914dc888bec03ce1276df8', 'request_id': 'req-28f39a0e-f4ed-4cd1-84a4-7ff272f21332'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:55.641032 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-28f39a0e-f4ed-4cd1-84a4-7ff272f21332 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da0a6f16bd914dc888bec03ce1276df8', 'request_id': 'req-28f39a0e-f4ed-4cd1-84a4-7ff272f21332'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:55.662026 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-28f39a0e-f4ed-4cd1-84a4-7ff272f21332 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da0a6f16bd914dc888bec03ce1276df8', 'request_id': 'req-28f39a0e-f4ed-4cd1-84a4-7ff272f21332'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:55.707279 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-28f39a0e-f4ed-4cd1-84a4-7ff272f21332 admin admin] The token cache is being invalidate because project da0a6f16bd914dc888bec03ce1276df8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:09:55.707530 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-28f39a0e-f4ed-4cd1-84a4-7ff272f21332 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:55.709278 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-28f39a0e-f4ed-4cd1-84a4-7ff272f21332 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:55.725091 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 685/1370] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:09:55 2022] DELETE /identity/v3/projects/da0a6f16bd914dc888bec03ce1276df8 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:09:55.731623 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7afaa46-7c5b-45f2-b206-228e17c63247 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:55.732639 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7afaa46-7c5b-45f2-b206-228e17c63247 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:55.732639 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7afaa46-7c5b-45f2-b206-228e17c63247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:55.778124 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231adcd2-32d3-4278-82ac-a1d5f27bf2f0 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:55.778124 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231adcd2-32d3-4278-82ac-a1d5f27bf2f0 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:55.778428 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231adcd2-32d3-4278-82ac-a1d5f27bf2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:55.814698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b7afaa46-7c5b-45f2-b206-228e17c63247 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:55.820639 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b7afaa46-7c5b-45f2-b206-228e17c63247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:55.863522 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-231adcd2-32d3-4278-82ac-a1d5f27bf2f0 None None] MFA Rules not processed for user `2e783caddcad473dbdd764919286e244`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:55.868393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-231adcd2-32d3-4278-82ac-a1d5f27bf2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:55.970716 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:56.016327 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 686/1371] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:56.024940 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa1c36-6223-471f-8807-1c25d9cd472f None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:56.025230 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa1c36-6223-471f-8807-1c25d9cd472f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:56.025543 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa1c36-6223-471f-8807-1c25d9cd472f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:56.048840 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 686/1372] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:56.061768 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231adcd2-32d3-4278-82ac-a1d5f27bf2f0 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:56.064645 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-231adcd2-32d3-4278-82ac-a1d5f27bf2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:56.075363 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-07aa1c36-6223-471f-8807-1c25d9cd472f None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:56.076606 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-07aa1c36-6223-471f-8807-1c25d9cd472f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:56.104226 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 687/1373] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:56.107428 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa1c36-6223-471f-8807-1c25d9cd472f None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:56.121941 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa1c36-6223-471f-8807-1c25d9cd472f None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:56.128739 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13a67f-d52b-438f-9209-9755b28d144a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:56.129681 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13a67f-d52b-438f-9209-9755b28d144a admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:56.130253 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13a67f-d52b-438f-9209-9755b28d144a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:56.131131 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13a67f-d52b-438f-9209-9755b28d144a admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:56.133405 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf13a67f-d52b-438f-9209-9755b28d144a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:09:56.135257 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf13a67f-d52b-438f-9209-9755b28d144a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:56.141355 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf13a67f-d52b-438f-9209-9755b28d144a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:56.160243 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 688/1374] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:09:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:09:56.164228 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13a67f-d52b-438f-9209-9755b28d144a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:56.177779 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13a67f-d52b-438f-9209-9755b28d144a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:56.184941 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be1352-84c0-4e8a-9469-10edb667e0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:56.185905 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be1352-84c0-4e8a-9469-10edb667e0f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:56.186340 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be1352-84c0-4e8a-9469-10edb667e0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:56.187499 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be1352-84c0-4e8a-9469-10edb667e0f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:56.188314 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72be1352-84c0-4e8a-9469-10edb667e0f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:56.192393 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72be1352-84c0-4e8a-9469-10edb667e0f8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:56.200881 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231adcd2-32d3-4278-82ac-a1d5f27bf2f0 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:56.207683 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4cf6e7-e43f-4b35-ab1e-40fb97362a14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:56.208437 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4cf6e7-e43f-4b35-ab1e-40fb97362a14 service nova] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:56.209327 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4cf6e7-e43f-4b35-ab1e-40fb97362a14 service nova] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:56.209327 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4cf6e7-e43f-4b35-ab1e-40fb97362a14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:56.218694 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4cf6e7-e43f-4b35-ab1e-40fb97362a14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:56.223668 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4cf6e7-e43f-4b35-ab1e-40fb97362a14 service nova] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:56.236812 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 687/1375] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:09:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:09:56.252401 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 689/1376] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:09:56 2022] POST /identity/v3/projects => generated 361 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:09:56.256494 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4cf6e7-e43f-4b35-ab1e-40fb97362a14 service nova] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:56.286832 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4cf6e7-e43f-4b35-ab1e-40fb97362a14 service nova] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:56.423608 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa8030-83f0-49db-965a-9db209798df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:56.423937 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa8030-83f0-49db-965a-9db209798df0 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:56.424125 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa8030-83f0-49db-965a-9db209798df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:56.424715 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa8030-83f0-49db-965a-9db209798df0 admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:56.425052 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa8030-83f0-49db-965a-9db209798df0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:56.429815 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa8030-83f0-49db-965a-9db209798df0 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:56.474713 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 688/1377] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:09:56 2022] POST /identity/v3/users => generated 281 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:09:56.478355 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be1352-84c0-4e8a-9469-10edb667e0f8 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:56.492823 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be1352-84c0-4e8a-9469-10edb667e0f8 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:56.524370 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610c1a2-e9e1-48a1-aacf-fe78601d2123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:56.525130 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610c1a2-e9e1-48a1-aacf-fe78601d2123 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:56.525550 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610c1a2-e9e1-48a1-aacf-fe78601d2123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:56.525953 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610c1a2-e9e1-48a1-aacf-fe78601d2123 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:56.526717 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610c1a2-e9e1-48a1-aacf-fe78601d2123 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:56.532346 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610c1a2-e9e1-48a1-aacf-fe78601d2123 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:56.550882 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa8030-83f0-49db-965a-9db209798df0 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:56.553582 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-acfa8030-83f0-49db-965a-9db209798df0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:56.562647 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 690/1378] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:09:56.563891 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610c1a2-e9e1-48a1-aacf-fe78601d2123 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:56.575884 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610c1a2-e9e1-48a1-aacf-fe78601d2123 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:56.587469 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeaf782-2c8c-4a43-8b36-c66ffa8e4bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:56.588280 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeaf782-2c8c-4a43-8b36-c66ffa8e4bc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:56.588659 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeaf782-2c8c-4a43-8b36-c66ffa8e4bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:56.589033 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeaf782-2c8c-4a43-8b36-c66ffa8e4bc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:56.589785 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeaf782-2c8c-4a43-8b36-c66ffa8e4bc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:56.591583 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeaf782-2c8c-4a43-8b36-c66ffa8e4bc4 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:56.591903 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa8030-83f0-49db-965a-9db209798df0 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:56.609070 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aac68e-1c96-4271-94e0-d192a61d06d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:56.609853 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aac68e-1c96-4271-94e0-d192a61d06d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:56.609988 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 691/1379] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:09:56.610107 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aac68e-1c96-4271-94e0-d192a61d06d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:56.610254 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aac68e-1c96-4271-94e0-d192a61d06d5 admin admin] PATH_INFO: `/v3/users/aa04de0ff2104dc9b39040fb11095d8e` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:56.611483 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeaf782-2c8c-4a43-8b36-c66ffa8e4bc4 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:56.612152 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aac68e-1c96-4271-94e0-d192a61d06d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa04de0ff2104dc9b39040fb11095d8e)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:56.614359 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aac68e-1c96-4271-94e0-d192a61d06d5 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:56.631956 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeaf782-2c8c-4a43-8b36-c66ffa8e4bc4 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:56.650310 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcb4c7-f991-4488-a555-94ca2940fa68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:56.651302 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcb4c7-f991-4488-a555-94ca2940fa68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:56.651531 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcb4c7-f991-4488-a555-94ca2940fa68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:56.651904 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcb4c7-f991-4488-a555-94ca2940fa68 admin admin] PATH_INFO: `/v3/projects/fd87874322b24343be605ca66356bfb0/users/a7c41f2af427492695af5a7c9df4f947/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:56.695589 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bcb4c7-f991-4488-a555-94ca2940fa68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd87874322b24343be605ca66356bfb0, user_id=a7c41f2af427492695af5a7c9df4f947, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:56.699020 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bcb4c7-f991-4488-a555-94ca2940fa68 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:56.726148 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-f2aac68e-1c96-4271-94e0-d192a61d06d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa04de0ff2104dc9b39040fb11095d8e'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:56.729492 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:56.732820 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 692/1380] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:09:56 2022] PUT /identity/v3/projects/fd87874322b24343be605ca66356bfb0/users/a7c41f2af427492695af5a7c9df4f947/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:09:56.745809 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d5ba7-3095-40b3-bd17-371c99abd77a None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:56.746174 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-f2aac68e-1c96-4271-94e0-d192a61d06d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa04de0ff2104dc9b39040fb11095d8e'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:56.746495 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d5ba7-3095-40b3-bd17-371c99abd77a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:56.746932 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d5ba7-3095-40b3-bd17-371c99abd77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:56.750086 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-f2aac68e-1c96-4271-94e0-d192a61d06d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa04de0ff2104dc9b39040fb11095d8e'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:56.751774 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-f2aac68e-1c96-4271-94e0-d192a61d06d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa04de0ff2104dc9b39040fb11095d8e'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:56.757672 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:56.778001 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-f2aac68e-1c96-4271-94e0-d192a61d06d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa04de0ff2104dc9b39040fb11095d8e'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:56.806453 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-419d5ba7-3095-40b3-bd17-371c99abd77a None None] MFA Rules not processed for user `a7c41f2af427492695af5a7c9df4f947`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:56.808438 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-419d5ba7-3095-40b3-bd17-371c99abd77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:56.824159 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 689/1381] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:09:56 2022] DELETE /identity/v3/users/aa04de0ff2104dc9b39040fb11095d8e => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:09:56.829575 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d8b8b-9981-461a-89a6-ecd66659ce20 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:56.830189 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d8b8b-9981-461a-89a6-ecd66659ce20 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:56.830189 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d8b8b-9981-461a-89a6-ecd66659ce20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:56.888196 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-d68d8b8b-9981-461a-89a6-ecd66659ce20 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:56.892007 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-d68d8b8b-9981-461a-89a6-ecd66659ce20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:57.019287 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 693/1382] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:57.023218 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d5ba7-3095-40b3-bd17-371c99abd77a None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:57.027261 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-419d5ba7-3095-40b3-bd17-371c99abd77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:57.077554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 690/1383] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:57.201025 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d8b8b-9981-461a-89a6-ecd66659ce20 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:57.203019 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-d68d8b8b-9981-461a-89a6-ecd66659ce20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:57.294756 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d5ba7-3095-40b3-bd17-371c99abd77a None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:57.307932 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd714078-24de-417e-a9d6-54556cf3ef0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:57.308566 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd714078-24de-417e-a9d6-54556cf3ef0f admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:57.308843 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd714078-24de-417e-a9d6-54556cf3ef0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:57.309113 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd714078-24de-417e-a9d6-54556cf3ef0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:57.309759 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd714078-24de-417e-a9d6-54556cf3ef0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:57.311709 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd714078-24de-417e-a9d6-54556cf3ef0f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:57.379137 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 694/1384] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:09:57 2022] POST /identity/v3/projects => generated 365 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:09:57.380722 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd714078-24de-417e-a9d6-54556cf3ef0f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:57.404435 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d8b8b-9981-461a-89a6-ecd66659ce20 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:57.524151 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ccad08-0356-4325-9894-27ba25849e32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:57.524967 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccad08-0356-4325-9894-27ba25849e32 service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:57.525901 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccad08-0356-4325-9894-27ba25849e32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:57.526284 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccad08-0356-4325-9894-27ba25849e32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:57.604023 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:57.611685 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd714078-24de-417e-a9d6-54556cf3ef0f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:57.630218 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386205a1-eb36-43b1-b0ff-aeeb09a148ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:57.635506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386205a1-eb36-43b1-b0ff-aeeb09a148ab admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:57.636174 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386205a1-eb36-43b1-b0ff-aeeb09a148ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:57.638593 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386205a1-eb36-43b1-b0ff-aeeb09a148ab admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:57.639564 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386205a1-eb36-43b1-b0ff-aeeb09a148ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:57.642902 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386205a1-eb36-43b1-b0ff-aeeb09a148ab admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:57.656329 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ccad08-0356-4325-9894-27ba25849e32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:57.658093 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ccad08-0356-4325-9894-27ba25849e32 service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:57.679756 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 691/1385] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:09:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:09:57.691984 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 695/1386] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:09:57 2022] POST /identity/v3/users => generated 289 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:09:57.695765 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ccad08-0356-4325-9894-27ba25849e32 service neutron] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:57.714762 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ccad08-0356-4325-9894-27ba25849e32 service neutron] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:57.724327 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43ff53-f78b-41da-b7ef-7e08cdf5bd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:57.725100 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43ff53-f78b-41da-b7ef-7e08cdf5bd28 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:57.725505 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43ff53-f78b-41da-b7ef-7e08cdf5bd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:57.725886 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43ff53-f78b-41da-b7ef-7e08cdf5bd28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:57.726641 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb43ff53-f78b-41da-b7ef-7e08cdf5bd28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:57.728566 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb43ff53-f78b-41da-b7ef-7e08cdf5bd28 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:57.752003 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 692/1387] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:09:57.755891 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386205a1-eb36-43b1-b0ff-aeeb09a148ab admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:57.770863 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386205a1-eb36-43b1-b0ff-aeeb09a148ab admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:57.789095 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2ecd3-a444-40c6-813b-3dd387b4d22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:57.791204 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2ecd3-a444-40c6-813b-3dd387b4d22d admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:57.792952 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2ecd3-a444-40c6-813b-3dd387b4d22d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:57.793419 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2ecd3-a444-40c6-813b-3dd387b4d22d admin admin] PATH_INFO: `/v3/projects/c0ab20fc330141d4bf7a5cd768e48cab/users/820b6e16685c46d9b4f68afdfcdc97dc/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:57.799493 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:57.829187 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2ecd3-a444-40c6-813b-3dd387b4d22d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0ab20fc330141d4bf7a5cd768e48cab, user_id=820b6e16685c46d9b4f68afdfcdc97dc, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:57.837270 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2ecd3-a444-40c6-813b-3dd387b4d22d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:57.868558 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 696/1388] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:09:57 2022] PUT /identity/v3/projects/c0ab20fc330141d4bf7a5cd768e48cab/users/820b6e16685c46d9b4f68afdfcdc97dc/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:09:57.870149 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43ff53-f78b-41da-b7ef-7e08cdf5bd28 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:57.897639 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43ff53-f78b-41da-b7ef-7e08cdf5bd28 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:58.042996 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2ecd3-a444-40c6-813b-3dd387b4d22d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:58.045312 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7cc2ecd3-a444-40c6-813b-3dd387b4d22d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:58.079762 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9202ca-ffc5-499d-8841-de1fb77fbb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:58.081437 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9202ca-ffc5-499d-8841-de1fb77fbb7f admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:58.082313 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9202ca-ffc5-499d-8841-de1fb77fbb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:58.084317 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9202ca-ffc5-499d-8841-de1fb77fbb7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:58.085733 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9202ca-ffc5-499d-8841-de1fb77fbb7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:58.091402 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9202ca-ffc5-499d-8841-de1fb77fbb7f admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:58.127786 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 693/1389] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:09:58.133791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9202ca-ffc5-499d-8841-de1fb77fbb7f admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:58.172908 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9202ca-ffc5-499d-8841-de1fb77fbb7f admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:58.191979 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180616f8-08bb-4a88-878a-51154bb32e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:58.194585 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180616f8-08bb-4a88-878a-51154bb32e7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:58.195520 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180616f8-08bb-4a88-878a-51154bb32e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:58.197770 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180616f8-08bb-4a88-878a-51154bb32e7e admin admin] PATH_INFO: `/v3/projects/c0ab20fc330141d4bf7a5cd768e48cab/users/820b6e16685c46d9b4f68afdfcdc97dc/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:58.205392 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180616f8-08bb-4a88-878a-51154bb32e7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0ab20fc330141d4bf7a5cd768e48cab, user_id=820b6e16685c46d9b4f68afdfcdc97dc, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:58.216347 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180616f8-08bb-4a88-878a-51154bb32e7e admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:58.250829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2ecd3-a444-40c6-813b-3dd387b4d22d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:58.257355 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 694/1390] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:09:58 2022] PUT /identity/v3/projects/c0ab20fc330141d4bf7a5cd768e48cab/users/820b6e16685c46d9b4f68afdfcdc97dc/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:09:58.270601 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e2af6-b0fa-4b8e-9c0e-7fc26685793b None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:58.270826 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e2af6-b0fa-4b8e-9c0e-7fc26685793b None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:58.271146 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e2af6-b0fa-4b8e-9c0e-7fc26685793b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:58.335707 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-146e2af6-b0fa-4b8e-9c0e-7fc26685793b None None] MFA Rules not processed for user `820b6e16685c46d9b4f68afdfcdc97dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:58.338327 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-146e2af6-b0fa-4b8e-9c0e-7fc26685793b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:58.397382 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b11ef-d01f-4da7-a9e6-e37c0d197aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:58.398287 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b11ef-d01f-4da7-a9e6-e37c0d197aeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:58.398838 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b11ef-d01f-4da7-a9e6-e37c0d197aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:58.399274 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b11ef-d01f-4da7-a9e6-e37c0d197aeb admin admin] PATH_INFO: `/v3/users/677a97f17a664152b8b79149e607550b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:58.400901 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b11ef-d01f-4da7-a9e6-e37c0d197aeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=677a97f17a664152b8b79149e607550b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:58.405485 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b11ef-d01f-4da7-a9e6-e37c0d197aeb admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:58.505510 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 695/1391] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:58 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:58.515388 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e2af6-b0fa-4b8e-9c0e-7fc26685793b None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:58.516840 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-146e2af6-b0fa-4b8e-9c0e-7fc26685793b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:58.523541 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0c6b11ef-d01f-4da7-a9e6-e37c0d197aeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '677a97f17a664152b8b79149e607550b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:58.537100 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0c6b11ef-d01f-4da7-a9e6-e37c0d197aeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '677a97f17a664152b8b79149e607550b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:58.539727 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0c6b11ef-d01f-4da7-a9e6-e37c0d197aeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '677a97f17a664152b8b79149e607550b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:58.558816 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0c6b11ef-d01f-4da7-a9e6-e37c0d197aeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '677a97f17a664152b8b79149e607550b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:58.580280 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0c6b11ef-d01f-4da7-a9e6-e37c0d197aeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '677a97f17a664152b8b79149e607550b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:09:58.595690 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 697/1392] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:09:58 2022] DELETE /identity/v3/users/677a97f17a664152b8b79149e607550b => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:09:58.599472 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851574de-47cc-4bc2-95e0-e695cc10e365 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:58.599763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851574de-47cc-4bc2-95e0-e695cc10e365 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:58.600188 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851574de-47cc-4bc2-95e0-e695cc10e365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:58.634932 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:58.667592 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-851574de-47cc-4bc2-95e0-e695cc10e365 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:58.671812 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-851574de-47cc-4bc2-95e0-e695cc10e365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:58.706998 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e2af6-b0fa-4b8e-9c0e-7fc26685793b None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:58.709046 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:09:58.843326 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6807f-38e3-49de-90c1-2ad62642a517 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:58.843948 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6807f-38e3-49de-90c1-2ad62642a517 service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:58.844226 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6807f-38e3-49de-90c1-2ad62642a517 service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:58.844496 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6807f-38e3-49de-90c1-2ad62642a517 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:58.848732 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-4ae6807f-38e3-49de-90c1-2ad62642a517 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:58.952138 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 698/1393] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:59.014185 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851574de-47cc-4bc2-95e0-e695cc10e365 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:59.015646 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-851574de-47cc-4bc2-95e0-e695cc10e365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:59.037238 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6807f-38e3-49de-90c1-2ad62642a517 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:59.038939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6807f-38e3-49de-90c1-2ad62642a517 service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:59.066463 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 696/1394] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:09:58 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:09:59.148249 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851574de-47cc-4bc2-95e0-e695cc10e365 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:09:59.164856 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe02138-b117-4d1f-bef4-9634051d3cea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:09:59.165495 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe02138-b117-4d1f-bef4-9634051d3cea service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:59.165815 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe02138-b117-4d1f-bef4-9634051d3cea service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:59.166096 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe02138-b117-4d1f-bef4-9634051d3cea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:59.186777 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe02138-b117-4d1f-bef4-9634051d3cea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:09:59.188346 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe02138-b117-4d1f-bef4-9634051d3cea service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:09:59.218743 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 699/1395] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:09:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 13 13:09:59.656191 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac22abc-b0d5-4d02-8c4b-0168bb267ebb None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:59.657131 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac22abc-b0d5-4d02-8c4b-0168bb267ebb None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:59.657668 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac22abc-b0d5-4d02-8c4b-0168bb267ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:59.720732 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-1ac22abc-b0d5-4d02-8c4b-0168bb267ebb None None] MFA Rules not processed for user `e3a049e77aba470db21e26d1bc8caa80`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:59.724318 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-1ac22abc-b0d5-4d02-8c4b-0168bb267ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:59.889316 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 697/1396] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:59.900298 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2cfec-fcdd-43b1-857a-c2986a440bc3 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:09:59.901025 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2cfec-fcdd-43b1-857a-c2986a440bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:09:59.901444 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2cfec-fcdd-43b1-857a-c2986a440bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:09:59.959810 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6be2cfec-fcdd-43b1-857a-c2986a440bc3 None None] MFA Rules not processed for user `e3a049e77aba470db21e26d1bc8caa80`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:09:59.960990 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6be2cfec-fcdd-43b1-857a-c2986a440bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:09:59.972394 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 700/1397] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:09:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:09:59.989349 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac22abc-b0d5-4d02-8c4b-0168bb267ebb None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:09:59.991464 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-1ac22abc-b0d5-4d02-8c4b-0168bb267ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:00.047436 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be2cfec-fcdd-43b1-857a-c2986a440bc3 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:00.050476 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6be2cfec-fcdd-43b1-857a-c2986a440bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:00.073478 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:00.081285 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be2cfec-fcdd-43b1-857a-c2986a440bc3 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:00.093482 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e2244-89a0-486f-9046-6722ae638012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:00.094441 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e2244-89a0-486f-9046-6722ae638012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:00.094725 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e2244-89a0-486f-9046-6722ae638012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:00.095077 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e2244-89a0-486f-9046-6722ae638012 admin admin] PATH_INFO: `/v3/users/a7c41f2af427492695af5a7c9df4f947` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:00.096331 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962e2244-89a0-486f-9046-6722ae638012 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7c41f2af427492695af5a7c9df4f947)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:00.101247 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962e2244-89a0-486f-9046-6722ae638012 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:00.158865 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac22abc-b0d5-4d02-8c4b-0168bb267ebb None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:00.167244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ce486-54e8-44a5-9eae-e690c48b2824 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:00.168743 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ce486-54e8-44a5-9eae-e690c48b2824 service nova] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:00.169298 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ce486-54e8-44a5-9eae-e690c48b2824 service nova] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:00.169705 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ce486-54e8-44a5-9eae-e690c48b2824 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:00.180630 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ce486-54e8-44a5-9eae-e690c48b2824 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:00.183588 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ce486-54e8-44a5-9eae-e690c48b2824 service nova] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:00.201528 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 698/1398] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:09:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:10:00.211726 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-962e2244-89a0-486f-9046-6722ae638012 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7c41f2af427492695af5a7c9df4f947'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:00.227618 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-962e2244-89a0-486f-9046-6722ae638012 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7c41f2af427492695af5a7c9df4f947'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:00.228378 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-962e2244-89a0-486f-9046-6722ae638012 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7c41f2af427492695af5a7c9df4f947'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:00.246676 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-962e2244-89a0-486f-9046-6722ae638012 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7c41f2af427492695af5a7c9df4f947'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:00.272383 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-962e2244-89a0-486f-9046-6722ae638012 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7c41f2af427492695af5a7c9df4f947'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:00.285311 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 701/1399] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:10:00 2022] DELETE /identity/v3/users/a7c41f2af427492695af5a7c9df4f947 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:00.290539 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1dfee-ddab-4e4a-9589-d0483ead9fd0 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:00.291124 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1dfee-ddab-4e4a-9589-d0483ead9fd0 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:00.291939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1dfee-ddab-4e4a-9589-d0483ead9fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:00.358611 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-d5a1dfee-ddab-4e4a-9589-d0483ead9fd0 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:00.369719 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-d5a1dfee-ddab-4e4a-9589-d0483ead9fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:00.538980 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e2244-89a0-486f-9046-6722ae638012 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:00.540617 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-962e2244-89a0-486f-9046-6722ae638012 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:00.620930 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:00.623488 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 699/1400] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:00.761726 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1dfee-ddab-4e4a-9589-d0483ead9fd0 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:00.763396 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-d5a1dfee-ddab-4e4a-9589-d0483ead9fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:00.852253 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e2244-89a0-486f-9046-6722ae638012 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:00.881753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0105fa-d12c-4c51-bbfd-1d028b3cbc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:00.882420 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0105fa-d12c-4c51-bbfd-1d028b3cbc48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:00.882703 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0105fa-d12c-4c51-bbfd-1d028b3cbc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:00.882971 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0105fa-d12c-4c51-bbfd-1d028b3cbc48 admin admin] PATH_INFO: `/v3/projects/0aff939a649349b89a6f166a47698373` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:00.884197 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0105fa-d12c-4c51-bbfd-1d028b3cbc48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0aff939a649349b89a6f166a47698373)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:00.886703 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0105fa-d12c-4c51-bbfd-1d028b3cbc48 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:00.953511 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1dfee-ddab-4e4a-9589-d0483ead9fd0 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:00.963321 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7993f9b-e18b-464a-abbd-320dc683ecaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:00.964083 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7993f9b-e18b-464a-abbd-320dc683ecaf service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:00.964580 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7993f9b-e18b-464a-abbd-320dc683ecaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:00.964854 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7993f9b-e18b-464a-abbd-320dc683ecaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:00.995303 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7993f9b-e18b-464a-abbd-320dc683ecaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:00.997943 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7993f9b-e18b-464a-abbd-320dc683ecaf service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:01.030690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 700/1401] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:10:01.071821 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ba0105fa-d12c-4c51-bbfd-1d028b3cbc48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0aff939a649349b89a6f166a47698373', 'request_id': 'req-ba0105fa-d12c-4c51-bbfd-1d028b3cbc48'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:01.088284 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ba0105fa-d12c-4c51-bbfd-1d028b3cbc48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0aff939a649349b89a6f166a47698373', 'request_id': 'req-ba0105fa-d12c-4c51-bbfd-1d028b3cbc48'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:01.116217 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ba0105fa-d12c-4c51-bbfd-1d028b3cbc48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0aff939a649349b89a6f166a47698373', 'request_id': 'req-ba0105fa-d12c-4c51-bbfd-1d028b3cbc48'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:01.137798 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ba0105fa-d12c-4c51-bbfd-1d028b3cbc48 admin admin] The token cache is being invalidate because project 0aff939a649349b89a6f166a47698373 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:10:01.138070 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ba0105fa-d12c-4c51-bbfd-1d028b3cbc48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:01.141985 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ba0105fa-d12c-4c51-bbfd-1d028b3cbc48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:01.148046 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 702/1402] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:10:00 2022] DELETE /identity/v3/projects/0aff939a649349b89a6f166a47698373 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:01.149757 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe9a62-dc02-49a2-a6b5-793f713289c3 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:01.150140 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe9a62-dc02-49a2-a6b5-793f713289c3 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:01.150519 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe9a62-dc02-49a2-a6b5-793f713289c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:01.208930 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-3abe9a62-dc02-49a2-a6b5-793f713289c3 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:01.210347 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-3abe9a62-dc02-49a2-a6b5-793f713289c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:01.368433 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:01.394479 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 701/1403] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:10:01.402399 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885120e-2b91-47d1-aa20-28ad3c7ba0ba None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:01.403495 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885120e-2b91-47d1-aa20-28ad3c7ba0ba None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:01.403918 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885120e-2b91-47d1-aa20-28ad3c7ba0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:01.453072 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-2885120e-2b91-47d1-aa20-28ad3c7ba0ba None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:01.454324 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2885120e-2b91-47d1-aa20-28ad3c7ba0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:01.486252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 703/1404] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:01.489903 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe9a62-dc02-49a2-a6b5-793f713289c3 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:01.503392 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe9a62-dc02-49a2-a6b5-793f713289c3 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:01.514453 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87567b88-7faa-443e-ab8f-c7c3abc73834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:01.515225 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87567b88-7faa-443e-ab8f-c7c3abc73834 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:01.515418 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87567b88-7faa-443e-ab8f-c7c3abc73834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:01.517546 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87567b88-7faa-443e-ab8f-c7c3abc73834 admin admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:01.518183 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87567b88-7faa-443e-ab8f-c7c3abc73834 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:10:01.518502 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87567b88-7faa-443e-ab8f-c7c3abc73834 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:01.520190 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87567b88-7faa-443e-ab8f-c7c3abc73834 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:01.550147 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 702/1405] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:10:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:10:01.555256 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885120e-2b91-47d1-aa20-28ad3c7ba0ba None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:01.574353 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885120e-2b91-47d1-aa20-28ad3c7ba0ba None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:01.580688 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2aa0f3-01cc-4493-8210-b42db94d6d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:01.581321 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2aa0f3-01cc-4493-8210-b42db94d6d1c admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:01.581604 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2aa0f3-01cc-4493-8210-b42db94d6d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:01.581878 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2aa0f3-01cc-4493-8210-b42db94d6d1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:01.582550 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2aa0f3-01cc-4493-8210-b42db94d6d1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:01.584495 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2aa0f3-01cc-4493-8210-b42db94d6d1c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:01.664272 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 704/1406] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:10:01 2022] POST /identity/v3/projects => generated 371 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:01.670604 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87567b88-7faa-443e-ab8f-c7c3abc73834 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:01.700911 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87567b88-7faa-443e-ab8f-c7c3abc73834 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:01.830822 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41927b56-cfbf-4ae5-a04f-1e80ed9b399a None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:01.831149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41927b56-cfbf-4ae5-a04f-1e80ed9b399a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:01.839534 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41927b56-cfbf-4ae5-a04f-1e80ed9b399a None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:01.841792 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 705/1407] 10.222.0.23 () {54 vars in 912 bytes} [Thu Oct 13 13:10:01 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:10:01.847764 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbc893-24a1-4130-955b-28ac72c7fd03 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:01.848051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbc893-24a1-4130-955b-28ac72c7fd03 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:01.848329 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbc893-24a1-4130-955b-28ac72c7fd03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:01.916818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386270f-3a54-4162-904c-d348621a9e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:01.918309 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386270f-3a54-4162-904c-d348621a9e6e admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:01.926363 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386270f-3a54-4162-904c-d348621a9e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:01.926858 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386270f-3a54-4162-904c-d348621a9e6e admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:01.928368 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a386270f-3a54-4162-904c-d348621a9e6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:01.930931 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a386270f-3a54-4162-904c-d348621a9e6e admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:01.978196 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 703/1408] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:10:01 2022] POST /identity/v3/users => generated 286 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:01.983762 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386270f-3a54-4162-904c-d348621a9e6e admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:02.039610 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386270f-3a54-4162-904c-d348621a9e6e admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:02.055611 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418037d-3334-40f4-93e3-cb594d64c6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:02.057214 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418037d-3334-40f4-93e3-cb594d64c6aa admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:02.059513 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418037d-3334-40f4-93e3-cb594d64c6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:02.060908 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418037d-3334-40f4-93e3-cb594d64c6aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:02.065052 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9418037d-3334-40f4-93e3-cb594d64c6aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:02.067005 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9418037d-3334-40f4-93e3-cb594d64c6aa admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:02.076127 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d7dbc893-24a1-4130-955b-28ac72c7fd03 None None] MFA Rules not processed for user `846251a661c04d3f9f195c4ac09b4189`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:02.078741 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d7dbc893-24a1-4130-955b-28ac72c7fd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:02.098550 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 704/1409] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:01 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:02.102471 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418037d-3334-40f4-93e3-cb594d64c6aa admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:02.128686 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418037d-3334-40f4-93e3-cb594d64c6aa admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:02.144990 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe82f7-81dd-4f82-9092-ee4cc50c12cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:02.145628 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe82f7-81dd-4f82-9092-ee4cc50c12cc admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:02.145911 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe82f7-81dd-4f82-9092-ee4cc50c12cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:02.146183 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe82f7-81dd-4f82-9092-ee4cc50c12cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:02.146839 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe82f7-81dd-4f82-9092-ee4cc50c12cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:02.154778 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe82f7-81dd-4f82-9092-ee4cc50c12cc admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:02.200005 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 705/1410] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:02.206643 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe82f7-81dd-4f82-9092-ee4cc50c12cc admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:02.242458 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe82f7-81dd-4f82-9092-ee4cc50c12cc admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:02.255804 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e865a-5d5d-4ba0-a3ea-18c005f761a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:02.256691 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e865a-5d5d-4ba0-a3ea-18c005f761a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:02.257141 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e865a-5d5d-4ba0-a3ea-18c005f761a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:02.257554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e865a-5d5d-4ba0-a3ea-18c005f761a8 admin admin] PATH_INFO: `/v3/projects/09f0359d0d60455ab08e2fd479a97cd8/users/bc7dfc9d2f5b405286aaf3ed23a14322/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:02.282816 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 706/1411] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Oct 13 13:10:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 436 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:10:02.290361 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7e865a-5d5d-4ba0-a3ea-18c005f761a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09f0359d0d60455ab08e2fd479a97cd8, user_id=bc7dfc9d2f5b405286aaf3ed23a14322, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:02.294213 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7e865a-5d5d-4ba0-a3ea-18c005f761a8 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:02.313544 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 706/1412] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:10:02 2022] PUT /identity/v3/projects/09f0359d0d60455ab08e2fd479a97cd8/users/bc7dfc9d2f5b405286aaf3ed23a14322/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:02.325858 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4069b7b-90da-4dc4-90c7-f41f131effeb None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:02.326492 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4069b7b-90da-4dc4-90c7-f41f131effeb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:02.326955 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4069b7b-90da-4dc4-90c7-f41f131effeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:02.333480 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e865a-5d5d-4ba0-a3ea-18c005f761a8 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:02.339148 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-ac7e865a-5d5d-4ba0-a3ea-18c005f761a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:02.413594 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f4069b7b-90da-4dc4-90c7-f41f131effeb None None] MFA Rules not processed for user `bc7dfc9d2f5b405286aaf3ed23a14322`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:02.419608 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f4069b7b-90da-4dc4-90c7-f41f131effeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:02.430895 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:02.566748 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e865a-5d5d-4ba0-a3ea-18c005f761a8 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:02.575829 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cece88f-d5d1-4ad7-b63b-31cfbc1f35f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:02.578364 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cece88f-d5d1-4ad7-b63b-31cfbc1f35f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:02.580791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cece88f-d5d1-4ad7-b63b-31cfbc1f35f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:02.581686 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cece88f-d5d1-4ad7-b63b-31cfbc1f35f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:02.617821 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cece88f-d5d1-4ad7-b63b-31cfbc1f35f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:02.623961 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cece88f-d5d1-4ad7-b63b-31cfbc1f35f1 service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:02.652632 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 707/1413] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:10:02.653601 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cece88f-d5d1-4ad7-b63b-31cfbc1f35f1 service neutron] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:02.659913 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-9cece88f-d5d1-4ad7-b63b-31cfbc1f35f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:02.689123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 707/1414] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:02.693299 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4069b7b-90da-4dc4-90c7-f41f131effeb None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:02.792869 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:02.865128 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cece88f-d5d1-4ad7-b63b-31cfbc1f35f1 service neutron] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:02.873180 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe36fc3-7e3c-45ee-944b-1b60148b6cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:02.873842 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe36fc3-7e3c-45ee-944b-1b60148b6cc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:02.874214 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe36fc3-7e3c-45ee-944b-1b60148b6cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:02.874539 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe36fc3-7e3c-45ee-944b-1b60148b6cc9 admin admin] PATH_INFO: `/v3/projects/4b4877965cc4405a9c793fde9f764e4d` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:02.875255 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4069b7b-90da-4dc4-90c7-f41f131effeb None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:02.875875 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe36fc3-7e3c-45ee-944b-1b60148b6cc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b4877965cc4405a9c793fde9f764e4d)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:02.878297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe36fc3-7e3c-45ee-944b-1b60148b6cc9 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:02.889604 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0938a0-3339-41c5-97c8-35e15c65460d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:02.890314 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0938a0-3339-41c5-97c8-35e15c65460d admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:02.890662 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0938a0-3339-41c5-97c8-35e15c65460d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:02.890946 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0938a0-3339-41c5-97c8-35e15c65460d admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:02.891716 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0938a0-3339-41c5-97c8-35e15c65460d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:02.899109 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0938a0-3339-41c5-97c8-35e15c65460d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:02.976957 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 708/1415] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:10:02 2022] POST /identity/v3/projects => generated 371 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:10:02.980728 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0938a0-3339-41c5-97c8-35e15c65460d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:03.004319 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-cfe36fc3-7e3c-45ee-944b-1b60148b6cc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b4877965cc4405a9c793fde9f764e4d', 'request_id': 'req-cfe36fc3-7e3c-45ee-944b-1b60148b6cc9'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:03.013202 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-cfe36fc3-7e3c-45ee-944b-1b60148b6cc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b4877965cc4405a9c793fde9f764e4d', 'request_id': 'req-cfe36fc3-7e3c-45ee-944b-1b60148b6cc9'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:03.022701 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-cfe36fc3-7e3c-45ee-944b-1b60148b6cc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b4877965cc4405a9c793fde9f764e4d', 'request_id': 'req-cfe36fc3-7e3c-45ee-944b-1b60148b6cc9'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:03.052719 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-cfe36fc3-7e3c-45ee-944b-1b60148b6cc9 admin admin] The token cache is being invalidate because project 4b4877965cc4405a9c793fde9f764e4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:10:03.053026 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-cfe36fc3-7e3c-45ee-944b-1b60148b6cc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:03.054517 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-cfe36fc3-7e3c-45ee-944b-1b60148b6cc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:03.057926 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 708/1416] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:10:02 2022] DELETE /identity/v3/projects/4b4877965cc4405a9c793fde9f764e4d => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:03.070095 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe36fc3-7e3c-45ee-944b-1b60148b6cc9 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:03.074525 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-cfe36fc3-7e3c-45ee-944b-1b60148b6cc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:03.117896 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0938a0-3339-41c5-97c8-35e15c65460d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:03.195513 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:03.219606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe36fc3-7e3c-45ee-944b-1b60148b6cc9 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:03.226488 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98736224-7fe6-4e45-9bc2-397cc97de957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:03.228388 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98736224-7fe6-4e45-9bc2-397cc97de957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:03.228671 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98736224-7fe6-4e45-9bc2-397cc97de957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:03.229384 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98736224-7fe6-4e45-9bc2-397cc97de957 admin admin] PATH_INFO: `/v3/users/e6428b84d2f94c18888dbb8008a20a1d` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:03.230602 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98736224-7fe6-4e45-9bc2-397cc97de957 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6428b84d2f94c18888dbb8008a20a1d)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:03.246091 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98736224-7fe6-4e45-9bc2-397cc97de957 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:03.276323 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07b266-3669-46b2-ad1e-f67b72a939d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:03.276984 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07b266-3669-46b2-ad1e-f67b72a939d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:03.277263 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07b266-3669-46b2-ad1e-f67b72a939d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:03.277534 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07b266-3669-46b2-ad1e-f67b72a939d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:03.278182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc07b266-3669-46b2-ad1e-f67b72a939d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:03.280165 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc07b266-3669-46b2-ad1e-f67b72a939d2 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:03.341832 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-98736224-7fe6-4e45-9bc2-397cc97de957 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6428b84d2f94c18888dbb8008a20a1d'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:03.349822 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 709/1417] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:10:02 2022] POST /identity/v3/users => generated 292 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:10:03.353748 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07b266-3669-46b2-ad1e-f67b72a939d2 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:03.357665 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-98736224-7fe6-4e45-9bc2-397cc97de957 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6428b84d2f94c18888dbb8008a20a1d'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:03.358413 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-98736224-7fe6-4e45-9bc2-397cc97de957 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6428b84d2f94c18888dbb8008a20a1d'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:03.359121 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-98736224-7fe6-4e45-9bc2-397cc97de957 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6428b84d2f94c18888dbb8008a20a1d'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:03.359933 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-cc07b266-3669-46b2-ad1e-f67b72a939d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:03.368145 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:03.379419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-98736224-7fe6-4e45-9bc2-397cc97de957 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6428b84d2f94c18888dbb8008a20a1d'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:03.397239 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07b266-3669-46b2-ad1e-f67b72a939d2 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:03.413627 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ecbedb-13de-4813-9658-6bb203afc0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:03.414261 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecbedb-13de-4813-9658-6bb203afc0de admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:03.414541 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecbedb-13de-4813-9658-6bb203afc0de admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:03.414811 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ecbedb-13de-4813-9658-6bb203afc0de admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:03.415504 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ecbedb-13de-4813-9658-6bb203afc0de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:03.417184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ecbedb-13de-4813-9658-6bb203afc0de admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:03.436680 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 709/1418] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:10:03 2022] DELETE /identity/v3/users/e6428b84d2f94c18888dbb8008a20a1d => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:03.448562 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 710/1419] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:03.450374 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98736224-7fe6-4e45-9bc2-397cc97de957 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:03.452380 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-98736224-7fe6-4e45-9bc2-397cc97de957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:03.545571 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96675e-f31d-48b2-bb0e-8f9b4c0891c3 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:03.546039 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96675e-f31d-48b2-bb0e-8f9b4c0891c3 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:03.546450 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96675e-f31d-48b2-bb0e-8f9b4c0891c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:03.576863 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-3c96675e-f31d-48b2-bb0e-8f9b4c0891c3 None None] MFA Rules not processed for user `2e783caddcad473dbdd764919286e244`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:03.578687 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3c96675e-f31d-48b2-bb0e-8f9b4c0891c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:03.598502 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98736224-7fe6-4e45-9bc2-397cc97de957 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:03.613079 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdff1fa-145a-42a1-812c-d65a7c4716c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:03.614099 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdff1fa-145a-42a1-812c-d65a7c4716c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:03.616607 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdff1fa-145a-42a1-812c-d65a7c4716c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:03.617304 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdff1fa-145a-42a1-812c-d65a7c4716c9 admin admin] PATH_INFO: `/v3/projects/b714f9e378a347bbb78cf393f2798dbe/users/2bb65b7d7c624e6bb1414020eac87271/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:03.667028 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdff1fa-145a-42a1-812c-d65a7c4716c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b714f9e378a347bbb78cf393f2798dbe, user_id=2bb65b7d7c624e6bb1414020eac87271, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:03.669943 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdff1fa-145a-42a1-812c-d65a7c4716c9 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:03.690789 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 711/1420] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:10:03.691871 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c96675e-f31d-48b2-bb0e-8f9b4c0891c3 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:03.697476 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3c96675e-f31d-48b2-bb0e-8f9b4c0891c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:03.699549 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 710/1421] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:10:03 2022] PUT /identity/v3/projects/b714f9e378a347bbb78cf393f2798dbe/users/2bb65b7d7c624e6bb1414020eac87271/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:03.703352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdff1fa-145a-42a1-812c-d65a7c4716c9 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:03.722866 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c96675e-f31d-48b2-bb0e-8f9b4c0891c3 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:03.876061 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b729990-4515-4cca-8596-39dfcf5b10d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:03.877093 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b729990-4515-4cca-8596-39dfcf5b10d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:03.877404 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b729990-4515-4cca-8596-39dfcf5b10d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:03.877680 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b729990-4515-4cca-8596-39dfcf5b10d6 admin admin] PATH_INFO: `/v3/projects/fd87874322b24343be605ca66356bfb0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:03.880424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b729990-4515-4cca-8596-39dfcf5b10d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd87874322b24343be605ca66356bfb0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:03.884806 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b729990-4515-4cca-8596-39dfcf5b10d6 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:03.924092 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdff1fa-145a-42a1-812c-d65a7c4716c9 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:03.935592 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c3c48-a5af-44a5-894b-b51bb4b3d980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:03.936520 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c3c48-a5af-44a5-894b-b51bb4b3d980 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:03.937017 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c3c48-a5af-44a5-894b-b51bb4b3d980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:03.937572 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c3c48-a5af-44a5-894b-b51bb4b3d980 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:03.938484 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1c3c48-a5af-44a5-894b-b51bb4b3d980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:03.940585 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1c3c48-a5af-44a5-894b-b51bb4b3d980 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:03.959726 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 711/1422] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:03.961348 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c3c48-a5af-44a5-894b-b51bb4b3d980 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:03.974479 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c3c48-a5af-44a5-894b-b51bb4b3d980 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:03.975762 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6b729990-4515-4cca-8596-39dfcf5b10d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd87874322b24343be605ca66356bfb0', 'request_id': 'req-6b729990-4515-4cca-8596-39dfcf5b10d6'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:03.999167 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6b729990-4515-4cca-8596-39dfcf5b10d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd87874322b24343be605ca66356bfb0', 'request_id': 'req-6b729990-4515-4cca-8596-39dfcf5b10d6'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:04.023570 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6b729990-4515-4cca-8596-39dfcf5b10d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd87874322b24343be605ca66356bfb0', 'request_id': 'req-6b729990-4515-4cca-8596-39dfcf5b10d6'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:04.027640 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:04.069074 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6b729990-4515-4cca-8596-39dfcf5b10d6 admin admin] The token cache is being invalidate because project fd87874322b24343be605ca66356bfb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:10:04.069375 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6b729990-4515-4cca-8596-39dfcf5b10d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:04.071119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6b729990-4515-4cca-8596-39dfcf5b10d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:04.075488 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 712/1423] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:10:03 2022] DELETE /identity/v3/projects/fd87874322b24343be605ca66356bfb0 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:04.079387 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b729990-4515-4cca-8596-39dfcf5b10d6 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:04.081241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6b729990-4515-4cca-8596-39dfcf5b10d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:04.100341 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:04.118103 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c1db6-fe51-4c24-b258-156aeca39052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:04.120183 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c1db6-fe51-4c24-b258-156aeca39052 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:04.121615 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c1db6-fe51-4c24-b258-156aeca39052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:04.121906 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c1db6-fe51-4c24-b258-156aeca39052 admin admin] PATH_INFO: `/v3/projects/b714f9e378a347bbb78cf393f2798dbe/users/2bb65b7d7c624e6bb1414020eac87271/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:04.129126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4c1db6-fe51-4c24-b258-156aeca39052 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b714f9e378a347bbb78cf393f2798dbe, user_id=2bb65b7d7c624e6bb1414020eac87271, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:04.135543 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4c1db6-fe51-4c24-b258-156aeca39052 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:04.186524 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 712/1424] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:10:03 2022] PUT /identity/v3/projects/b714f9e378a347bbb78cf393f2798dbe/users/2bb65b7d7c624e6bb1414020eac87271/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:04.190264 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5f6ce-a2bd-4546-8997-874442a8e0e9 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:04.190598 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5f6ce-a2bd-4546-8997-874442a8e0e9 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:04.190879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5f6ce-a2bd-4546-8997-874442a8e0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:04.248791 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b729990-4515-4cca-8596-39dfcf5b10d6 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:04.300475 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-45e5f6ce-a2bd-4546-8997-874442a8e0e9 None None] MFA Rules not processed for user `2bb65b7d7c624e6bb1414020eac87271`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:04.302220 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-45e5f6ce-a2bd-4546-8997-874442a8e0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:04.428356 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba84709f-171a-4b3a-a007-4bd97d07a2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:04.429480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84709f-171a-4b3a-a007-4bd97d07a2a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:04.429849 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84709f-171a-4b3a-a007-4bd97d07a2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:04.431704 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84709f-171a-4b3a-a007-4bd97d07a2a6 admin admin] PATH_INFO: `/v3/users/820b6e16685c46d9b4f68afdfcdc97dc` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:04.434029 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba84709f-171a-4b3a-a007-4bd97d07a2a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=820b6e16685c46d9b4f68afdfcdc97dc)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:04.436232 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba84709f-171a-4b3a-a007-4bd97d07a2a6 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:04.529662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 713/1425] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:04 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:04.536320 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e5f6ce-a2bd-4546-8997-874442a8e0e9 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:04.545676 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-45e5f6ce-a2bd-4546-8997-874442a8e0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:04.594171 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ba84709f-171a-4b3a-a007-4bd97d07a2a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '820b6e16685c46d9b4f68afdfcdc97dc'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:04.612998 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ba84709f-171a-4b3a-a007-4bd97d07a2a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '820b6e16685c46d9b4f68afdfcdc97dc'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:04.614865 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ba84709f-171a-4b3a-a007-4bd97d07a2a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '820b6e16685c46d9b4f68afdfcdc97dc'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:04.641912 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ba84709f-171a-4b3a-a007-4bd97d07a2a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '820b6e16685c46d9b4f68afdfcdc97dc'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:04.690388 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ba84709f-171a-4b3a-a007-4bd97d07a2a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '820b6e16685c46d9b4f68afdfcdc97dc'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:04.704262 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e5f6ce-a2bd-4546-8997-874442a8e0e9 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:04.711389 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a8b14-2f61-496f-9115-49e92974641f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:04.712252 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a8b14-2f61-496f-9115-49e92974641f service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:04.713826 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a8b14-2f61-496f-9115-49e92974641f service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:04.714170 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a8b14-2f61-496f-9115-49e92974641f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:04.721187 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-229a8b14-2f61-496f-9115-49e92974641f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:04.721347 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 713/1426] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:10:04 2022] DELETE /identity/v3/users/820b6e16685c46d9b4f68afdfcdc97dc => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:10:04.722683 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d24a59-be4d-4a7f-880c-5320c103ee89 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:04.727993 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d24a59-be4d-4a7f-880c-5320c103ee89 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:04.732498 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d24a59-be4d-4a7f-880c-5320c103ee89 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:04.740137 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 714/1427] 10.222.0.23 () {54 vars in 912 bytes} [Thu Oct 13 13:10:04 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:10:04.742437 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8703a-cbcf-47f5-a327-c259ef33bd11 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:04.754540 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8703a-cbcf-47f5-a327-c259ef33bd11 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:04.757424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8703a-cbcf-47f5-a327-c259ef33bd11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:04.778291 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229a8b14-2f61-496f-9115-49e92974641f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:04.780096 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229a8b14-2f61-496f-9115-49e92974641f service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:04.829718 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-74c8703a-cbcf-47f5-a327-c259ef33bd11 None None] MFA Rules not processed for user `454203f47b434cca952a1cef56de559f`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:04.831525 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-74c8703a-cbcf-47f5-a327-c259ef33bd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:04.841877 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:04.857284 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 714/1428] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:04 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:10:04.858470 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a8b14-2f61-496f-9115-49e92974641f service cinder] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:04.863556 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-229a8b14-2f61-496f-9115-49e92974641f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:04.921183 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a8b14-2f61-496f-9115-49e92974641f service cinder] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:04.957915 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 715/1429] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:04 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:04.959098 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2fa6e2-6231-45a8-acf5-3f42929e4a45 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:04.959653 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2fa6e2-6231-45a8-acf5-3f42929e4a45 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:04.959925 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2fa6e2-6231-45a8-acf5-3f42929e4a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:04.991014 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6c2fa6e2-6231-45a8-acf5-3f42929e4a45 None None] MFA Rules not processed for user `846251a661c04d3f9f195c4ac09b4189`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:04.995434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6c2fa6e2-6231-45a8-acf5-3f42929e4a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:05.041854 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f0a2d-8bef-4af1-b6b9-0f3688fea8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:05.042401 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f0a2d-8bef-4af1-b6b9-0f3688fea8e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:05.042688 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f0a2d-8bef-4af1-b6b9-0f3688fea8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:05.042961 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f0a2d-8bef-4af1-b6b9-0f3688fea8e1 admin admin] PATH_INFO: `/v3/projects/20a7329f9d8d42e29484421023854c1c` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:05.047182 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f0a2d-8bef-4af1-b6b9-0f3688fea8e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20a7329f9d8d42e29484421023854c1c)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:05.049338 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f0a2d-8bef-4af1-b6b9-0f3688fea8e1 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:05.084435 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:05.095355 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 716/1430] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Oct 13 13:10:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:05.096660 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2fa6e2-6231-45a8-acf5-3f42929e4a45 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:05.098709 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6c2fa6e2-6231-45a8-acf5-3f42929e4a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:05.137011 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-da4f0a2d-8bef-4af1-b6b9-0f3688fea8e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20a7329f9d8d42e29484421023854c1c', 'request_id': 'req-da4f0a2d-8bef-4af1-b6b9-0f3688fea8e1'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:05.145170 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-da4f0a2d-8bef-4af1-b6b9-0f3688fea8e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20a7329f9d8d42e29484421023854c1c', 'request_id': 'req-da4f0a2d-8bef-4af1-b6b9-0f3688fea8e1'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:05.156723 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-da4f0a2d-8bef-4af1-b6b9-0f3688fea8e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20a7329f9d8d42e29484421023854c1c', 'request_id': 'req-da4f0a2d-8bef-4af1-b6b9-0f3688fea8e1'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:05.182093 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-da4f0a2d-8bef-4af1-b6b9-0f3688fea8e1 admin admin] The token cache is being invalidate because project 20a7329f9d8d42e29484421023854c1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:10:05.182500 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-da4f0a2d-8bef-4af1-b6b9-0f3688fea8e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:05.183227 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-da4f0a2d-8bef-4af1-b6b9-0f3688fea8e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:05.188087 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 715/1431] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:10:04 2022] DELETE /identity/v3/projects/20a7329f9d8d42e29484421023854c1c => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:05.188087 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f60c82-25fd-4cb5-8a6f-75a8b95858bd None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:05.188087 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f60c82-25fd-4cb5-8a6f-75a8b95858bd None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:05.188311 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f60c82-25fd-4cb5-8a6f-75a8b95858bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:05.204053 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2fa6e2-6231-45a8-acf5-3f42929e4a45 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:05.279830 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-a6f60c82-25fd-4cb5-8a6f-75a8b95858bd None None] MFA Rules not processed for user `b5dbaf53464b44fe8d9e16f5013587b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:05.281535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-a6f60c82-25fd-4cb5-8a6f-75a8b95858bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:05.344980 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:05.372520 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdef9b1-d208-417d-9b16-a99db5b599c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a971cdb7f9d4e6692c6326c103d7303', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:05.375627 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdef9b1-d208-417d-9b16-a99db5b599c8 service placement] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:05.378402 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdef9b1-d208-417d-9b16-a99db5b599c8 service placement] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:05.378970 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdef9b1-d208-417d-9b16-a99db5b599c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:05.382410 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8fdef9b1-d208-417d-9b16-a99db5b599c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:05.417987 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 716/1432] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:05 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:05.419335 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a3210-0b9f-45cd-b884-8dd2de7e89d3 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:05.419624 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a3210-0b9f-45cd-b884-8dd2de7e89d3 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:05.419900 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a3210-0b9f-45cd-b884-8dd2de7e89d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:05.421178 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdef9b1-d208-417d-9b16-a99db5b599c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:05.422999 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdef9b1-d208-417d-9b16-a99db5b599c8 service placement] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:05.446614 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 717/1433] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:10:05.449460 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdef9b1-d208-417d-9b16-a99db5b599c8 service placement] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:05.467968 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8fdef9b1-d208-417d-9b16-a99db5b599c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:05.521362 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-233a3210-0b9f-45cd-b884-8dd2de7e89d3 None None] MFA Rules not processed for user `454203f47b434cca952a1cef56de559f`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:05.530217 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-233a3210-0b9f-45cd-b884-8dd2de7e89d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:05.696384 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdef9b1-d208-417d-9b16-a99db5b599c8 service placement] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:05.705689 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a87f43a-a57f-4336-8243-f6418ae01b62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:05.706592 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87f43a-a57f-4336-8243-f6418ae01b62 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:05.708136 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87f43a-a57f-4336-8243-f6418ae01b62 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:05.709231 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a87f43a-a57f-4336-8243-f6418ae01b62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:05.713292 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7a87f43a-a57f-4336-8243-f6418ae01b62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:05.730227 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 717/1434] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:05.737024 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a3210-0b9f-45cd-b884-8dd2de7e89d3 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:05.744093 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-233a3210-0b9f-45cd-b884-8dd2de7e89d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:05.852293 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:05.876290 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a87f43a-a57f-4336-8243-f6418ae01b62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:05.878752 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a87f43a-a57f-4336-8243-f6418ae01b62 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:05.892850 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 718/1435] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:10:05.894677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a87f43a-a57f-4336-8243-f6418ae01b62 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:05.896347 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7a87f43a-a57f-4336-8243-f6418ae01b62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:05.906243 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a3210-0b9f-45cd-b884-8dd2de7e89d3 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:05.918071 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca429cc-410a-4afc-bf3a-40b61c2a840f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:05.921518 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca429cc-410a-4afc-bf3a-40b61c2a840f service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:05.922042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca429cc-410a-4afc-bf3a-40b61c2a840f service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:05.922517 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca429cc-410a-4afc-bf3a-40b61c2a840f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:05.926332 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-4ca429cc-410a-4afc-bf3a-40b61c2a840f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:05.953391 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca429cc-410a-4afc-bf3a-40b61c2a840f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:05.956165 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca429cc-410a-4afc-bf3a-40b61c2a840f service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:05.986445 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 718/1436] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:10:05.987658 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca429cc-410a-4afc-bf3a-40b61c2a840f service neutron] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:06.000335 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-4ca429cc-410a-4afc-bf3a-40b61c2a840f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:06.035289 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a87f43a-a57f-4336-8243-f6418ae01b62 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:06.040827 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0835abe-7c68-4de4-9726-b24721f74417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:06.041622 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0835abe-7c68-4de4-9726-b24721f74417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:06.042010 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0835abe-7c68-4de4-9726-b24721f74417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:06.042774 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0835abe-7c68-4de4-9726-b24721f74417 admin admin] PATH_INFO: `/v3/users/0f8b329b1b944bf78c97edbb2b7c8ec5` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:06.044107 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0835abe-7c68-4de4-9726-b24721f74417 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f8b329b1b944bf78c97edbb2b7c8ec5)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:06.046569 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0835abe-7c68-4de4-9726-b24721f74417 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:06.153865 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:06.160533 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a0835abe-7c68-4de4-9726-b24721f74417 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f8b329b1b944bf78c97edbb2b7c8ec5'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:06.192984 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a0835abe-7c68-4de4-9726-b24721f74417 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f8b329b1b944bf78c97edbb2b7c8ec5'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:06.193899 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a0835abe-7c68-4de4-9726-b24721f74417 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f8b329b1b944bf78c97edbb2b7c8ec5'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:06.212490 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a0835abe-7c68-4de4-9726-b24721f74417 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f8b329b1b944bf78c97edbb2b7c8ec5'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:06.251280 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a0835abe-7c68-4de4-9726-b24721f74417 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f8b329b1b944bf78c97edbb2b7c8ec5'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:06.268951 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 719/1437] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:10:05 2022] DELETE /identity/v3/users/0f8b329b1b944bf78c97edbb2b7c8ec5 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:06.279295 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0835abe-7c68-4de4-9726-b24721f74417 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:06.283982 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a0835abe-7c68-4de4-9726-b24721f74417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:06.289070 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca429cc-410a-4afc-bf3a-40b61c2a840f service neutron] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:06.430574 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0835abe-7c68-4de4-9726-b24721f74417 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:06.441838 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5022a731-3c74-4755-8654-61df53080819 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:06.443315 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022a731-3c74-4755-8654-61df53080819 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:06.443573 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022a731-3c74-4755-8654-61df53080819 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:06.443858 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022a731-3c74-4755-8654-61df53080819 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:06.446402 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5022a731-3c74-4755-8654-61df53080819 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:06.504616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4cb80f-494f-4bd3-8d05-7f5d30b39dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:06.505736 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4cb80f-494f-4bd3-8d05-7f5d30b39dd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:06.506196 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4cb80f-494f-4bd3-8d05-7f5d30b39dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:06.506669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4cb80f-494f-4bd3-8d05-7f5d30b39dd6 admin admin] PATH_INFO: `/v3/projects/c0ab20fc330141d4bf7a5cd768e48cab` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:06.510366 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4cb80f-494f-4bd3-8d05-7f5d30b39dd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0ab20fc330141d4bf7a5cd768e48cab)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:06.512622 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4cb80f-494f-4bd3-8d05-7f5d30b39dd6 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:06.590221 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5022a731-3c74-4755-8654-61df53080819 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:06.591849 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5022a731-3c74-4755-8654-61df53080819 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:06.623829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 720/1438] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:06 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:10:06.626093 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5022a731-3c74-4755-8654-61df53080819 service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:06.629303 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5022a731-3c74-4755-8654-61df53080819 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:06.651716 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-dd4cb80f-494f-4bd3-8d05-7f5d30b39dd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0ab20fc330141d4bf7a5cd768e48cab', 'request_id': 'req-dd4cb80f-494f-4bd3-8d05-7f5d30b39dd6'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:06.671904 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-dd4cb80f-494f-4bd3-8d05-7f5d30b39dd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0ab20fc330141d4bf7a5cd768e48cab', 'request_id': 'req-dd4cb80f-494f-4bd3-8d05-7f5d30b39dd6'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:06.679689 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5022a731-3c74-4755-8654-61df53080819 service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:06.687852 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-dd4cb80f-494f-4bd3-8d05-7f5d30b39dd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0ab20fc330141d4bf7a5cd768e48cab', 'request_id': 'req-dd4cb80f-494f-4bd3-8d05-7f5d30b39dd6'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:06.713969 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-dd4cb80f-494f-4bd3-8d05-7f5d30b39dd6 admin admin] The token cache is being invalidate because project c0ab20fc330141d4bf7a5cd768e48cab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:10:06.714386 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-dd4cb80f-494f-4bd3-8d05-7f5d30b39dd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:06.715450 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-dd4cb80f-494f-4bd3-8d05-7f5d30b39dd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:06.721665 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 719/1439] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:10:05 2022] DELETE /identity/v3/projects/c0ab20fc330141d4bf7a5cd768e48cab => generated 0 bytes in 731 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:06.723493 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dcbccb-492a-4b72-bfdc-57cb1cde14ef None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:06.723891 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dcbccb-492a-4b72-bfdc-57cb1cde14ef None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:06.724274 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dcbccb-492a-4b72-bfdc-57cb1cde14ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:06.787216 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-d1dcbccb-492a-4b72-bfdc-57cb1cde14ef None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:06.794888 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-d1dcbccb-492a-4b72-bfdc-57cb1cde14ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:06.865966 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:06.870998 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716949f-2429-449f-9aae-9a554c3dc712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:06.876441 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716949f-2429-449f-9aae-9a554c3dc712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:06.876803 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716949f-2429-449f-9aae-9a554c3dc712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:06.877090 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716949f-2429-449f-9aae-9a554c3dc712 admin admin] PATH_INFO: `/v3/users/da29e093d8bd48f781d5f7874869c572` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:06.878663 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6716949f-2429-449f-9aae-9a554c3dc712 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da29e093d8bd48f781d5f7874869c572)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:06.881901 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6716949f-2429-449f-9aae-9a554c3dc712 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:06.885546 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:07.020085 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 720/1440] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:07.022324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dcbccb-492a-4b72-bfdc-57cb1cde14ef None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:07.027457 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-d1dcbccb-492a-4b72-bfdc-57cb1cde14ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:07.055209 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6716949f-2429-449f-9aae-9a554c3dc712 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da29e093d8bd48f781d5f7874869c572'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:07.072228 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6716949f-2429-449f-9aae-9a554c3dc712 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da29e093d8bd48f781d5f7874869c572'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:07.073032 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6716949f-2429-449f-9aae-9a554c3dc712 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da29e093d8bd48f781d5f7874869c572'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:07.089851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6716949f-2429-449f-9aae-9a554c3dc712 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da29e093d8bd48f781d5f7874869c572'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:07.112556 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6716949f-2429-449f-9aae-9a554c3dc712 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da29e093d8bd48f781d5f7874869c572'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:07.139348 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 721/1441] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:10:06 2022] DELETE /identity/v3/users/da29e093d8bd48f781d5f7874869c572 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:07.144390 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8571d-61ff-4503-85eb-7b11b43d00c9 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:07.144688 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8571d-61ff-4503-85eb-7b11b43d00c9 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:07.144969 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8571d-61ff-4503-85eb-7b11b43d00c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:07.225960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d0d8571d-61ff-4503-85eb-7b11b43d00c9 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:07.227241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d0d8571d-61ff-4503-85eb-7b11b43d00c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:07.312432 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dcbccb-492a-4b72-bfdc-57cb1cde14ef None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:07.494619 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 722/1442] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:07.496153 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d7b78-ad34-4847-bb16-306dca0781d9 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:07.496402 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d7b78-ad34-4847-bb16-306dca0781d9 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:07.496612 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d7b78-ad34-4847-bb16-306dca0781d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:07.573035 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8c3d7b78-ad34-4847-bb16-306dca0781d9 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:07.577411 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8c3d7b78-ad34-4847-bb16-306dca0781d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:07.579820 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038b4df-ea71-4603-8574-b4149ca16514 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:07.580422 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038b4df-ea71-4603-8574-b4149ca16514 service nova] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:07.583762 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038b4df-ea71-4603-8574-b4149ca16514 service nova] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:07.584048 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d038b4df-ea71-4603-8574-b4149ca16514 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:07.595287 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-d038b4df-ea71-4603-8574-b4149ca16514 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:07.643156 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 723/1443] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:07.646460 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d7b78-ad34-4847-bb16-306dca0781d9 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:07.651487 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8c3d7b78-ad34-4847-bb16-306dca0781d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:07.706068 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d7b78-ad34-4847-bb16-306dca0781d9 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:07.744884 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a85d7-b798-4606-bf28-929f8f2c3a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:07.749157 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a85d7-b798-4606-bf28-929f8f2c3a3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:07.749157 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a85d7-b798-4606-bf28-929f8f2c3a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:07.749458 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a85d7-b798-4606-bf28-929f8f2c3a3a admin admin] PATH_INFO: `/v3/projects/eaeb18573403411d86a50298a48330d5` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:07.757413 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a85d7-b798-4606-bf28-929f8f2c3a3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaeb18573403411d86a50298a48330d5)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:07.771539 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a85d7-b798-4606-bf28-929f8f2c3a3a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:07.911000 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d038b4df-ea71-4603-8574-b4149ca16514 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:07.914567 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d038b4df-ea71-4603-8574-b4149ca16514 service nova] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:07.939261 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:07.960333 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b89a85d7-b798-4606-bf28-929f8f2c3a3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaeb18573403411d86a50298a48330d5', 'request_id': 'req-b89a85d7-b798-4606-bf28-929f8f2c3a3a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:07.980758 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 721/1444] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:07 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 965 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:10:07.981323 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b89a85d7-b798-4606-bf28-929f8f2c3a3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaeb18573403411d86a50298a48330d5', 'request_id': 'req-b89a85d7-b798-4606-bf28-929f8f2c3a3a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:07.983770 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038b4df-ea71-4603-8574-b4149ca16514 service nova] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:07.996301 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b89a85d7-b798-4606-bf28-929f8f2c3a3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaeb18573403411d86a50298a48330d5', 'request_id': 'req-b89a85d7-b798-4606-bf28-929f8f2c3a3a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:08.008335 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:08.023662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d038b4df-ea71-4603-8574-b4149ca16514 service nova] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:08.028941 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b89a85d7-b798-4606-bf28-929f8f2c3a3a admin admin] The token cache is being invalidate because project eaeb18573403411d86a50298a48330d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:10:08.032032 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b89a85d7-b798-4606-bf28-929f8f2c3a3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:08.033907 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b89a85d7-b798-4606-bf28-929f8f2c3a3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:08.037054 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 724/1445] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:10:07 2022] DELETE /identity/v3/projects/eaeb18573403411d86a50298a48330d5 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:08.051752 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a85d7-b798-4606-bf28-929f8f2c3a3a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:08.069956 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b89a85d7-b798-4606-bf28-929f8f2c3a3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:08.252503 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a85d7-b798-4606-bf28-929f8f2c3a3a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:08.264470 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e241e890-b944-45d3-bfb5-c0d983e7375b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:08.265110 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e241e890-b944-45d3-bfb5-c0d983e7375b service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:08.265392 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e241e890-b944-45d3-bfb5-c0d983e7375b service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:08.265668 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e241e890-b944-45d3-bfb5-c0d983e7375b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:08.269119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e241e890-b944-45d3-bfb5-c0d983e7375b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:08.513769 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e241e890-b944-45d3-bfb5-c0d983e7375b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:08.515239 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e241e890-b944-45d3-bfb5-c0d983e7375b service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:08.521846 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28efd3-7907-43ed-9682-95c6bd4d6579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:08.522939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28efd3-7907-43ed-9682-95c6bd4d6579 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:08.523368 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28efd3-7907-43ed-9682-95c6bd4d6579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:08.523642 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28efd3-7907-43ed-9682-95c6bd4d6579 admin admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:08.524361 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28efd3-7907-43ed-9682-95c6bd4d6579 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:10:08.524767 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28efd3-7907-43ed-9682-95c6bd4d6579 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:08.526806 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28efd3-7907-43ed-9682-95c6bd4d6579 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:08.548820 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 722/1446] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:10:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 567 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:10:08.550867 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0b140-a718-42de-ae5a-28c6113e8f1a None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:08.551403 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0b140-a718-42de-ae5a-28c6113e8f1a None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:08.552133 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0b140-a718-42de-ae5a-28c6113e8f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:08.555457 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 725/1447] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:10:08.557701 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e241e890-b944-45d3-bfb5-c0d983e7375b service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:08.568845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e241e890-b944-45d3-bfb5-c0d983e7375b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:08.592928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-e7f0b140-a718-42de-ae5a-28c6113e8f1a None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:08.596004 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-e7f0b140-a718-42de-ae5a-28c6113e8f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:08.613666 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e241e890-b944-45d3-bfb5-c0d983e7375b service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:08.618996 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 723/1448] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:08.627306 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20de177-61e4-4dd9-9546-2df9e2e255ff None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:08.627589 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20de177-61e4-4dd9-9546-2df9e2e255ff None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:08.627853 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20de177-61e4-4dd9-9546-2df9e2e255ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:08.641741 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865ccd3-f47e-475f-a090-3f0470085e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:08.642143 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865ccd3-f47e-475f-a090-3f0470085e50 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:08.642507 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865ccd3-f47e-475f-a090-3f0470085e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:08.642701 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865ccd3-f47e-475f-a090-3f0470085e50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:08.643391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6865ccd3-f47e-475f-a090-3f0470085e50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:08.654888 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6865ccd3-f47e-475f-a090-3f0470085e50 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:08.681562 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-a20de177-61e4-4dd9-9546-2df9e2e255ff None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:08.688009 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-a20de177-61e4-4dd9-9546-2df9e2e255ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:08.721400 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 724/1449] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:10:08.725909 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20de177-61e4-4dd9-9546-2df9e2e255ff None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:08.751129 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20de177-61e4-4dd9-9546-2df9e2e255ff None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:08.760632 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 726/1450] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:10:08 2022] POST /identity/v3/projects => generated 355 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:08.764188 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865ccd3-f47e-475f-a090-3f0470085e50 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:08.767987 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f83030-00bb-43c1-a271-d59ae7060426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:08.769842 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f83030-00bb-43c1-a271-d59ae7060426 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:08.771215 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f83030-00bb-43c1-a271-d59ae7060426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:08.771560 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f83030-00bb-43c1-a271-d59ae7060426 admin admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:08.773181 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f83030-00bb-43c1-a271-d59ae7060426 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:10:08.773869 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f83030-00bb-43c1-a271-d59ae7060426 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:08.778445 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f83030-00bb-43c1-a271-d59ae7060426 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:08.813898 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 725/1451] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 13:10:08.818167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f83030-00bb-43c1-a271-d59ae7060426 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:08.844364 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f83030-00bb-43c1-a271-d59ae7060426 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:08.966026 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865ccd3-f47e-475f-a090-3f0470085e50 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:08.979012 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4838eed3-e2e5-461b-9dd2-d3a6b217cfa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:08.979819 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838eed3-e2e5-461b-9dd2-d3a6b217cfa5 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:08.980238 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838eed3-e2e5-461b-9dd2-d3a6b217cfa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:08.980740 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838eed3-e2e5-461b-9dd2-d3a6b217cfa5 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:08.981602 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4838eed3-e2e5-461b-9dd2-d3a6b217cfa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:08.986670 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4838eed3-e2e5-461b-9dd2-d3a6b217cfa5 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:09.050844 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 727/1452] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:10:08 2022] POST /identity/v3/users => generated 278 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:09.056438 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4838eed3-e2e5-461b-9dd2-d3a6b217cfa5 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:09.080404 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2720b6f-3711-4521-8b77-8f5e494f00a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:09.081078 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2720b6f-3711-4521-8b77-8f5e494f00a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:09.085235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2720b6f-3711-4521-8b77-8f5e494f00a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:09.085235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2720b6f-3711-4521-8b77-8f5e494f00a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:09.085235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2720b6f-3711-4521-8b77-8f5e494f00a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:09.085235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2720b6f-3711-4521-8b77-8f5e494f00a6 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:09.096611 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4838eed3-e2e5-461b-9dd2-d3a6b217cfa5 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:09.120202 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b67cb1-0e76-47e7-9c9f-bed7b113983a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:09.121100 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b67cb1-0e76-47e7-9c9f-bed7b113983a admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:09.121573 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b67cb1-0e76-47e7-9c9f-bed7b113983a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:09.122080 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b67cb1-0e76-47e7-9c9f-bed7b113983a admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:09.122996 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b67cb1-0e76-47e7-9c9f-bed7b113983a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:09.125578 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b67cb1-0e76-47e7-9c9f-bed7b113983a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:09.139813 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 726/1453] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:10:08 2022] POST /identity/v3/projects => generated 385 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:09.140949 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2720b6f-3711-4521-8b77-8f5e494f00a6 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:09.165190 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 728/1454] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:09 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:09.180263 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b67cb1-0e76-47e7-9c9f-bed7b113983a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:09.204291 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:09.221806 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b67cb1-0e76-47e7-9c9f-bed7b113983a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:09.348277 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:09.412749 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2720b6f-3711-4521-8b77-8f5e494f00a6 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:09.443319 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dced7a-1b36-4439-a755-35681a6350f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:09.443319 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dced7a-1b36-4439-a755-35681a6350f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:09.443319 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dced7a-1b36-4439-a755-35681a6350f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:09.443319 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dced7a-1b36-4439-a755-35681a6350f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:09.446608 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dced7a-1b36-4439-a755-35681a6350f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:09.449573 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dced7a-1b36-4439-a755-35681a6350f0 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:09.452060 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a39a29-350c-4334-b264-7f4ebe58376c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:09.453184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a39a29-350c-4334-b264-7f4ebe58376c admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:09.453476 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a39a29-350c-4334-b264-7f4ebe58376c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:09.453759 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a39a29-350c-4334-b264-7f4ebe58376c admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:09.458698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a39a29-350c-4334-b264-7f4ebe58376c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:09.465829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a39a29-350c-4334-b264-7f4ebe58376c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:09.489770 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 729/1455] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:09 2022] GET /identity/v3/roles => generated 1420 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:09.493642 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fcb896-45c8-479a-af06-8c4478282b7d None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:09.494423 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fcb896-45c8-479a-af06-8c4478282b7d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:09.495668 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fcb896-45c8-479a-af06-8c4478282b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:09.510909 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 727/1456] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:10:09 2022] POST /identity/v3/users => generated 293 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:09.512027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dced7a-1b36-4439-a755-35681a6350f0 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:09.528832 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dced7a-1b36-4439-a755-35681a6350f0 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:09.542743 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4b6fa-f827-4ed2-abbf-a7d03df6a159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:09.544033 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4b6fa-f827-4ed2-abbf-a7d03df6a159 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:09.545758 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4b6fa-f827-4ed2-abbf-a7d03df6a159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:09.547099 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4b6fa-f827-4ed2-abbf-a7d03df6a159 admin admin] PATH_INFO: `/v3/projects/a197a3a289f94205b4395a33e8dc1b0f/users/f6bfa8fad6e04c05b2c7fe8b01ab9bf1/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:09.576583 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced4b6fa-f827-4ed2-abbf-a7d03df6a159 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a197a3a289f94205b4395a33e8dc1b0f, user_id=f6bfa8fad6e04c05b2c7fe8b01ab9bf1, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:09.581628 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced4b6fa-f827-4ed2-abbf-a7d03df6a159 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:09.630116 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-e9fcb896-45c8-479a-af06-8c4478282b7d None None] MFA Rules not processed for user `b6ae674f0efc445fa970fca64e13893a`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:09.630869 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 728/1457] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:10:09 2022] PUT /identity/v3/projects/a197a3a289f94205b4395a33e8dc1b0f/users/f6bfa8fad6e04c05b2c7fe8b01ab9bf1/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:09.632936 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4b6fa-f827-4ed2-abbf-a7d03df6a159 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:09.635264 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e9fcb896-45c8-479a-af06-8c4478282b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:09.860509 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4b6fa-f827-4ed2-abbf-a7d03df6a159 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:09.866183 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 730/1458] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Oct 13 13:10:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:09.869279 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9546950-8eb6-4981-b69b-573f7362bdf3 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:09.870082 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9546950-8eb6-4981-b69b-573f7362bdf3 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:09.871217 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9546950-8eb6-4981-b69b-573f7362bdf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:09.874983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3999c37e-a040-4dfc-8c31-be93675a0d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:09.876452 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999c37e-a040-4dfc-8c31-be93675a0d1f admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:09.876737 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999c37e-a040-4dfc-8c31-be93675a0d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:09.877014 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999c37e-a040-4dfc-8c31-be93675a0d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:09.877675 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3999c37e-a040-4dfc-8c31-be93675a0d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:09.879675 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3999c37e-a040-4dfc-8c31-be93675a0d1f admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:09.909584 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 729/1459] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:09 2022] GET /identity/v3/roles => generated 1420 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:09.911431 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7d77c-d2dc-41d5-84b5-8ecbe6f6e3a8 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:09.913975 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7d77c-d2dc-41d5-84b5-8ecbe6f6e3a8 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:09.914238 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7d77c-d2dc-41d5-84b5-8ecbe6f6e3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:09.943622 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-e9546950-8eb6-4981-b69b-573f7362bdf3 None None] MFA Rules not processed for user `f6bfa8fad6e04c05b2c7fe8b01ab9bf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:09.948610 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e9546950-8eb6-4981-b69b-573f7362bdf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:09.970850 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-67c7d77c-d2dc-41d5-84b5-8ecbe6f6e3a8 None None] MFA Rules not processed for user `454203f47b434cca952a1cef56de559f`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:09.976784 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-67c7d77c-d2dc-41d5-84b5-8ecbe6f6e3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:10.170958 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:10.173203 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 730/1460] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:09 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:10.188494 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7d77c-d2dc-41d5-84b5-8ecbe6f6e3a8 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:10.195788 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 731/1461] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:09 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:10.195788 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9546950-8eb6-4981-b69b-573f7362bdf3 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:10.199461 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e9546950-8eb6-4981-b69b-573f7362bdf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:10.221284 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7d77c-d2dc-41d5-84b5-8ecbe6f6e3a8 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:10.255125 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92421724-0507-4257-901f-93c3e68ed9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:10.257672 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92421724-0507-4257-901f-93c3e68ed9bc admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:10.257982 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92421724-0507-4257-901f-93c3e68ed9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:10.258229 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92421724-0507-4257-901f-93c3e68ed9bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:10.258909 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92421724-0507-4257-901f-93c3e68ed9bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:10.272566 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92421724-0507-4257-901f-93c3e68ed9bc admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:10.315134 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 731/1462] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:10 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:10.316240 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92421724-0507-4257-901f-93c3e68ed9bc admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:10.340354 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92421724-0507-4257-901f-93c3e68ed9bc admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:10.355546 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50733e9-7baa-43d4-ad27-b2aaca0991a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:10.356440 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50733e9-7baa-43d4-ad27-b2aaca0991a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:10.357429 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50733e9-7baa-43d4-ad27-b2aaca0991a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:10.359709 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50733e9-7baa-43d4-ad27-b2aaca0991a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:10.361682 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50733e9-7baa-43d4-ad27-b2aaca0991a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:10.368468 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50733e9-7baa-43d4-ad27-b2aaca0991a0 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:10.438974 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 732/1463] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:10:10 2022] POST /identity/v3/projects => generated 357 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:10.438974 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50733e9-7baa-43d4-ad27-b2aaca0991a0 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:10.487602 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9546950-8eb6-4981-b69b-573f7362bdf3 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:10.533322 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:10.648838 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50733e9-7baa-43d4-ad27-b2aaca0991a0 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:10.663957 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3263e-8830-4ec8-9325-29aca4edf51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:10.664645 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3263e-8830-4ec8-9325-29aca4edf51e admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:10.664924 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3263e-8830-4ec8-9325-29aca4edf51e admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:10.665194 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3263e-8830-4ec8-9325-29aca4edf51e admin admin] PATH_INFO: `/v3/projects/7e54a1d888f74dd683f52403d6f93fce/users/b33a8b1f4b63461c9f78a046c1c41b77/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:10.682229 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2582cf-9f91-4751-b65f-f0adde85e2dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526000341fd146db8c661f1fd14d4801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:10.682911 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2582cf-9f91-4751-b65f-f0adde85e2dd service swift] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:10.683163 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2582cf-9f91-4751-b65f-f0adde85e2dd service swift] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:10.683436 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2582cf-9f91-4751-b65f-f0adde85e2dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:10.703398 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f3263e-8830-4ec8-9325-29aca4edf51e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e54a1d888f74dd683f52403d6f93fce, user_id=b33a8b1f4b63461c9f78a046c1c41b77, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:10.706013 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f3263e-8830-4ec8-9325-29aca4edf51e admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:10.733627 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 733/1464] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:10:10 2022] PUT /identity/v3/projects/7e54a1d888f74dd683f52403d6f93fce/users/b33a8b1f4b63461c9f78a046c1c41b77/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:10.746796 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3263e-8830-4ec8-9325-29aca4edf51e admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:10.834253 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2582cf-9f91-4751-b65f-f0adde85e2dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:10.835953 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2582cf-9f91-4751-b65f-f0adde85e2dd service swift] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:10.855323 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 732/1465] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 13 13:10:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 13 13:10:10.856773 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb69c25-343f-4cbe-b47e-26c147ccfa7e None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:10.856953 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb69c25-343f-4cbe-b47e-26c147ccfa7e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:10.857321 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb69c25-343f-4cbe-b47e-26c147ccfa7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:10.957046 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-bdb69c25-343f-4cbe-b47e-26c147ccfa7e None None] MFA Rules not processed for user `b33a8b1f4b63461c9f78a046c1c41b77`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:10.958394 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-bdb69c25-343f-4cbe-b47e-26c147ccfa7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:10.969802 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3263e-8830-4ec8-9325-29aca4edf51e admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:10.996487 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:11.023972 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d053dca-7d3f-43f9-8fec-b5fbecc8714a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:11.039675 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d053dca-7d3f-43f9-8fec-b5fbecc8714a admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:11.042580 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d053dca-7d3f-43f9-8fec-b5fbecc8714a admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:11.043742 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d053dca-7d3f-43f9-8fec-b5fbecc8714a admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:11.046904 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d053dca-7d3f-43f9-8fec-b5fbecc8714a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:11.059009 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d053dca-7d3f-43f9-8fec-b5fbecc8714a admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:11.141917 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 734/1466] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:10:10 2022] POST /identity/v3/users => generated 285 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:11.146089 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d053dca-7d3f-43f9-8fec-b5fbecc8714a admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:11.174015 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d053dca-7d3f-43f9-8fec-b5fbecc8714a admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:11.191111 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e09d6c-f9cb-4a98-9855-80e06f542ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:11.191951 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e09d6c-f9cb-4a98-9855-80e06f542ebf admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:11.192509 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e09d6c-f9cb-4a98-9855-80e06f542ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:11.192882 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e09d6c-f9cb-4a98-9855-80e06f542ebf admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:11.193776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e09d6c-f9cb-4a98-9855-80e06f542ebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:11.194134 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 733/1467] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:11.195878 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb69c25-343f-4cbe-b47e-26c147ccfa7e None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:11.197914 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e09d6c-f9cb-4a98-9855-80e06f542ebf admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:11.199570 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-bdb69c25-343f-4cbe-b47e-26c147ccfa7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:11.240021 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb69c25-343f-4cbe-b47e-26c147ccfa7e None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:11.250519 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a0f58-d105-41d0-9aeb-798c6fc9e274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:11.251439 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a0f58-d105-41d0-9aeb-798c6fc9e274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:11.251855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a0f58-d105-41d0-9aeb-798c6fc9e274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:11.252262 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a0f58-d105-41d0-9aeb-798c6fc9e274 admin admin] PATH_INFO: `/v3/projects/157902d713ed4996a65d061aba4e4312` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:11.258515 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1a0f58-d105-41d0-9aeb-798c6fc9e274 admin admin] RBAC: Authorizing `identity:delete_project(project_id=157902d713ed4996a65d061aba4e4312)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:11.260727 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1a0f58-d105-41d0-9aeb-798c6fc9e274 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:11.265708 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 735/1468] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:11 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:10:11.268666 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e09d6c-f9cb-4a98-9855-80e06f542ebf admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:11.296791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e09d6c-f9cb-4a98-9855-80e06f542ebf admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:11.308220 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e73d56-7c86-4946-8a72-f263911bd960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:11.309111 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e73d56-7c86-4946-8a72-f263911bd960 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:11.309592 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e73d56-7c86-4946-8a72-f263911bd960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:11.310009 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e73d56-7c86-4946-8a72-f263911bd960 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:11.310878 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e73d56-7c86-4946-8a72-f263911bd960 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:11.315422 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e73d56-7c86-4946-8a72-f263911bd960 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:11.374301 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 736/1469] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:10:11 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:11.374301 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e73d56-7c86-4946-8a72-f263911bd960 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:11.428042 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4f1a0f58-d105-41d0-9aeb-798c6fc9e274 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '157902d713ed4996a65d061aba4e4312', 'request_id': 'req-4f1a0f58-d105-41d0-9aeb-798c6fc9e274'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:11.441522 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4f1a0f58-d105-41d0-9aeb-798c6fc9e274 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '157902d713ed4996a65d061aba4e4312', 'request_id': 'req-4f1a0f58-d105-41d0-9aeb-798c6fc9e274'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:11.455463 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4f1a0f58-d105-41d0-9aeb-798c6fc9e274 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '157902d713ed4996a65d061aba4e4312', 'request_id': 'req-4f1a0f58-d105-41d0-9aeb-798c6fc9e274'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:11.481513 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4f1a0f58-d105-41d0-9aeb-798c6fc9e274 admin admin] The token cache is being invalidate because project 157902d713ed4996a65d061aba4e4312 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:10:11.481809 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4f1a0f58-d105-41d0-9aeb-798c6fc9e274 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:11.484839 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4f1a0f58-d105-41d0-9aeb-798c6fc9e274 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:11.487737 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 734/1470] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:10:11 2022] DELETE /identity/v3/projects/157902d713ed4996a65d061aba4e4312 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:11.488073 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a0f58-d105-41d0-9aeb-798c6fc9e274 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:11.490023 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4f1a0f58-d105-41d0-9aeb-798c6fc9e274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:11.518397 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e73d56-7c86-4946-8a72-f263911bd960 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:11.544106 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:11.669149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a0f58-d105-41d0-9aeb-798c6fc9e274 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:11.676770 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab1865-ce5b-4b5e-a18b-4270b00c367d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:11.677669 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab1865-ce5b-4b5e-a18b-4270b00c367d admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:11.677948 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab1865-ce5b-4b5e-a18b-4270b00c367d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:11.678234 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab1865-ce5b-4b5e-a18b-4270b00c367d admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:11.678866 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab1865-ce5b-4b5e-a18b-4270b00c367d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:11.686143 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f97c03-b2fb-4216-9040-d4bd1408977f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:11.686867 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f97c03-b2fb-4216-9040-d4bd1408977f admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:11.687237 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f97c03-b2fb-4216-9040-d4bd1408977f admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:11.687717 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f97c03-b2fb-4216-9040-d4bd1408977f admin admin] PATH_INFO: `/v3/projects/ffa7f718542d4898abc2b8d5fcbe7bf1/users/283f83f1bb2f4eb0b3f77418fd6d0333/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:11.692734 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab1865-ce5b-4b5e-a18b-4270b00c367d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:11.716064 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f97c03-b2fb-4216-9040-d4bd1408977f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffa7f718542d4898abc2b8d5fcbe7bf1, user_id=283f83f1bb2f4eb0b3f77418fd6d0333, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:11.723222 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f97c03-b2fb-4216-9040-d4bd1408977f admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:11.726767 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 735/1471] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:10:11 2022] POST /identity/v3/users => generated 300 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:11.728454 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab1865-ce5b-4b5e-a18b-4270b00c367d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:11.739650 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-01ab1865-ce5b-4b5e-a18b-4270b00c367d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:11.751206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 737/1472] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:10:11 2022] PUT /identity/v3/projects/ffa7f718542d4898abc2b8d5fcbe7bf1/users/283f83f1bb2f4eb0b3f77418fd6d0333/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:11.752056 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f97c03-b2fb-4216-9040-d4bd1408977f admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:11.769996 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab1865-ce5b-4b5e-a18b-4270b00c367d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:11.844903 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:11.878332 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f97c03-b2fb-4216-9040-d4bd1408977f admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:11.890846 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44805a8a-878c-4e98-912e-54fb4e920fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:11.892173 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44805a8a-878c-4e98-912e-54fb4e920fb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:11.892968 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44805a8a-878c-4e98-912e-54fb4e920fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:11.893311 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44805a8a-878c-4e98-912e-54fb4e920fb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:11.894582 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44805a8a-878c-4e98-912e-54fb4e920fb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:11.896892 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44805a8a-878c-4e98-912e-54fb4e920fb4 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:11.927715 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 738/1473] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:11 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:11.927715 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44805a8a-878c-4e98-912e-54fb4e920fb4 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:11.931783 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-44805a8a-878c-4e98-912e-54fb4e920fb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:11.955602 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44805a8a-878c-4e98-912e-54fb4e920fb4 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:11.963073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa234c4-4970-4802-adf0-87dc87eb84b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:11.963950 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa234c4-4970-4802-adf0-87dc87eb84b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:11.964175 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa234c4-4970-4802-adf0-87dc87eb84b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:11.964492 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa234c4-4970-4802-adf0-87dc87eb84b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:11.965157 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa234c4-4970-4802-adf0-87dc87eb84b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:11.966785 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa234c4-4970-4802-adf0-87dc87eb84b9 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:11.982138 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 739/1474] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:11 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:11.982977 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa234c4-4970-4802-adf0-87dc87eb84b9 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:11.987201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5871c-cb3b-41bd-858c-645539b9c05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:11.987846 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5871c-cb3b-41bd-858c-645539b9c05e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:11.988120 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5871c-cb3b-41bd-858c-645539b9c05e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:11.988391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5871c-cb3b-41bd-858c-645539b9c05e admin admin] PATH_INFO: `/v3/users/8f001ff5529f4598b5ebd9502ab25176` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:11.993679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa234c4-4970-4802-adf0-87dc87eb84b9 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:11.995890 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f5871c-cb3b-41bd-858c-645539b9c05e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f001ff5529f4598b5ebd9502ab25176)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:11.997985 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f5871c-cb3b-41bd-858c-645539b9c05e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:11.998801 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad741c5b-0e9a-4d15-b3db-427f96c1a305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:11.999770 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad741c5b-0e9a-4d15-b3db-427f96c1a305 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:12.000135 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad741c5b-0e9a-4d15-b3db-427f96c1a305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:12.000481 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad741c5b-0e9a-4d15-b3db-427f96c1a305 admin admin] PATH_INFO: `/v3/projects/15acee4862cc4fdfab62c492de2c55b2/users/b4462e9ad1b0467498edd185c4c20557/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:12.065329 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad741c5b-0e9a-4d15-b3db-427f96c1a305 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15acee4862cc4fdfab62c492de2c55b2, user_id=b4462e9ad1b0467498edd185c4c20557, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:12.070516 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad741c5b-0e9a-4d15-b3db-427f96c1a305 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:12.094688 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 740/1475] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:10:11 2022] PUT /identity/v3/projects/15acee4862cc4fdfab62c492de2c55b2/users/b4462e9ad1b0467498edd185c4c20557/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:12.094688 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad741c5b-0e9a-4d15-b3db-427f96c1a305 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:12.124744 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-06f5871c-cb3b-41bd-858c-645539b9c05e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f001ff5529f4598b5ebd9502ab25176'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:12.136881 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-06f5871c-cb3b-41bd-858c-645539b9c05e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f001ff5529f4598b5ebd9502ab25176'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:12.139101 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-06f5871c-cb3b-41bd-858c-645539b9c05e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f001ff5529f4598b5ebd9502ab25176'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:12.159212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-06f5871c-cb3b-41bd-858c-645539b9c05e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f001ff5529f4598b5ebd9502ab25176'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:12.203757 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-06f5871c-cb3b-41bd-858c-645539b9c05e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f001ff5529f4598b5ebd9502ab25176'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:12.221006 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 736/1476] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:10:11 2022] DELETE /identity/v3/users/8f001ff5529f4598b5ebd9502ab25176 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:12.223195 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5871c-cb3b-41bd-858c-645539b9c05e admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:12.226317 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-06f5871c-cb3b-41bd-858c-645539b9c05e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:12.298123 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:12.306811 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad741c5b-0e9a-4d15-b3db-427f96c1a305 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:12.427709 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef5b1b-5389-40f7-ac10-d3fb16ebb5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:12.429191 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef5b1b-5389-40f7-ac10-d3fb16ebb5de admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:12.432145 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef5b1b-5389-40f7-ac10-d3fb16ebb5de admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:12.433001 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef5b1b-5389-40f7-ac10-d3fb16ebb5de admin admin] PATH_INFO: `/v3/projects/ffa7f718542d4898abc2b8d5fcbe7bf1/users/283f83f1bb2f4eb0b3f77418fd6d0333/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:12.438829 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ef5b1b-5389-40f7-ac10-d3fb16ebb5de admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffa7f718542d4898abc2b8d5fcbe7bf1, user_id=283f83f1bb2f4eb0b3f77418fd6d0333, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:12.441427 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ef5b1b-5389-40f7-ac10-d3fb16ebb5de admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:12.462586 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 741/1477] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:10:12 2022] PUT /identity/v3/projects/ffa7f718542d4898abc2b8d5fcbe7bf1/users/283f83f1bb2f4eb0b3f77418fd6d0333/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:12.464415 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2baf4ba-afbc-40dc-afa4-a4256d790ec8 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:12.466113 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2baf4ba-afbc-40dc-afa4-a4256d790ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:12.466113 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2baf4ba-afbc-40dc-afa4-a4256d790ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:12.470676 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5871c-cb3b-41bd-858c-645539b9c05e admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:12.524536 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-f2baf4ba-afbc-40dc-afa4-a4256d790ec8 None None] MFA Rules not processed for user `283f83f1bb2f4eb0b3f77418fd6d0333`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:12.525784 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-f2baf4ba-afbc-40dc-afa4-a4256d790ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:12.531592 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:12.556464 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ac3e7-6291-4e3a-9be1-adef2b7f78d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:12.557581 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ac3e7-6291-4e3a-9be1-adef2b7f78d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:12.558022 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ac3e7-6291-4e3a-9be1-adef2b7f78d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:12.558396 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ac3e7-6291-4e3a-9be1-adef2b7f78d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:12.560564 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ac3e7-6291-4e3a-9be1-adef2b7f78d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:12.563625 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ac3e7-6291-4e3a-9be1-adef2b7f78d8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:12.579139 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 737/1478] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:12 2022] GET /identity/v3/roles => generated 1420 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:12.585261 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ac3e7-6291-4e3a-9be1-adef2b7f78d8 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:12.605368 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ac3e7-6291-4e3a-9be1-adef2b7f78d8 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:12.612734 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a20c2a-1e2a-4b4b-8a81-b804f05b50ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:12.614671 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a20c2a-1e2a-4b4b-8a81-b804f05b50ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:12.615200 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a20c2a-1e2a-4b4b-8a81-b804f05b50ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:12.615578 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a20c2a-1e2a-4b4b-8a81-b804f05b50ad admin admin] PATH_INFO: `/v3/projects/15acee4862cc4fdfab62c492de2c55b2/users/b4462e9ad1b0467498edd185c4c20557/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:12.620897 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a20c2a-1e2a-4b4b-8a81-b804f05b50ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=15acee4862cc4fdfab62c492de2c55b2, user_id=b4462e9ad1b0467498edd185c4c20557, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:12.645388 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a20c2a-1e2a-4b4b-8a81-b804f05b50ad admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:12.663715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 738/1479] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:10:12 2022] PUT /identity/v3/projects/15acee4862cc4fdfab62c492de2c55b2/users/b4462e9ad1b0467498edd185c4c20557/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:12.678858 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9add308-3aee-4c0e-a942-8cce5e089dd7 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:12.679533 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9add308-3aee-4c0e-a942-8cce5e089dd7 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:12.679926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9add308-3aee-4c0e-a942-8cce5e089dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:12.716136 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 742/1480] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:12 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:12.726066 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2baf4ba-afbc-40dc-afa4-a4256d790ec8 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:12.728712 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-f2baf4ba-afbc-40dc-afa4-a4256d790ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:12.793669 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f9add308-3aee-4c0e-a942-8cce5e089dd7 None None] MFA Rules not processed for user `b4462e9ad1b0467498edd185c4c20557`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:12.795885 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f9add308-3aee-4c0e-a942-8cce5e089dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:12.955696 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2baf4ba-afbc-40dc-afa4-a4256d790ec8 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:13.191148 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 739/1481] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:12 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 512 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:13.192206 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9add308-3aee-4c0e-a942-8cce5e089dd7 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:13.200870 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f9add308-3aee-4c0e-a942-8cce5e089dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:13.203455 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:13.268605 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f634e1a-cee5-4d26-af40-4d5f0c19b2eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:13.271903 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f634e1a-cee5-4d26-af40-4d5f0c19b2eb service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:13.272886 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f634e1a-cee5-4d26-af40-4d5f0c19b2eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:13.273702 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f634e1a-cee5-4d26-af40-4d5f0c19b2eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:13.465215 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9add308-3aee-4c0e-a942-8cce5e089dd7 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:13.476869 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd79914-0461-4b4b-bc65-5c4b13a7864b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:13.479510 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd79914-0461-4b4b-bc65-5c4b13a7864b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:13.479802 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd79914-0461-4b4b-bc65-5c4b13a7864b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:13.480075 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd79914-0461-4b4b-bc65-5c4b13a7864b admin admin] PATH_INFO: `/v3/projects/dae34432cef2481e861e1ea3ce6dad77` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:13.487239 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd79914-0461-4b4b-bc65-5c4b13a7864b admin admin] RBAC: Authorizing `identity:delete_project(project_id=dae34432cef2481e861e1ea3ce6dad77)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:13.489342 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd79914-0461-4b4b-bc65-5c4b13a7864b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:13.578025 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f634e1a-cee5-4d26-af40-4d5f0c19b2eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:13.603273 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3bd79914-0461-4b4b-bc65-5c4b13a7864b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dae34432cef2481e861e1ea3ce6dad77', 'request_id': 'req-3bd79914-0461-4b4b-bc65-5c4b13a7864b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:13.603702 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f634e1a-cee5-4d26-af40-4d5f0c19b2eb service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:13.609280 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3bd79914-0461-4b4b-bc65-5c4b13a7864b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dae34432cef2481e861e1ea3ce6dad77', 'request_id': 'req-3bd79914-0461-4b4b-bc65-5c4b13a7864b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:13.627788 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3bd79914-0461-4b4b-bc65-5c4b13a7864b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dae34432cef2481e861e1ea3ce6dad77', 'request_id': 'req-3bd79914-0461-4b4b-bc65-5c4b13a7864b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:13.629699 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:13.653549 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3bd79914-0461-4b4b-bc65-5c4b13a7864b admin admin] The token cache is being invalidate because project dae34432cef2481e861e1ea3ce6dad77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:10:13.653974 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3bd79914-0461-4b4b-bc65-5c4b13a7864b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:13.655456 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 743/1482] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:12 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 930 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:10:13.655735 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3bd79914-0461-4b4b-bc65-5c4b13a7864b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:13.661748 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 740/1483] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:10:13 2022] DELETE /identity/v3/projects/dae34432cef2481e861e1ea3ce6dad77 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:13.663160 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd79914-0461-4b4b-bc65-5c4b13a7864b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:13.668556 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3bd79914-0461-4b4b-bc65-5c4b13a7864b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:13.674806 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56712c-63a8-408f-9715-af3ac0d99d56 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:13.677646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56712c-63a8-408f-9715-af3ac0d99d56 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:13.682973 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56712c-63a8-408f-9715-af3ac0d99d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:13.764987 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-8b56712c-63a8-408f-9715-af3ac0d99d56 None None] MFA Rules not processed for user `846251a661c04d3f9f195c4ac09b4189`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:13.767042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-8b56712c-63a8-408f-9715-af3ac0d99d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:13.771874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd79914-0461-4b4b-bc65-5c4b13a7864b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:13.787746 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a932d-0b98-424b-8390-f9a2cc81e788 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:13.788551 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a932d-0b98-424b-8390-f9a2cc81e788 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:13.789125 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a932d-0b98-424b-8390-f9a2cc81e788 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:13.789403 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a932d-0b98-424b-8390-f9a2cc81e788 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:13.793212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-af0a932d-0b98-424b-8390-f9a2cc81e788 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:13.948612 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 744/1484] 10.222.0.50 () {58 vars in 1015 bytes} [Thu Oct 13 13:10:13 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:13.950207 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3958e-ab2f-4c89-8593-c0993d03a23b None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:13.950602 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3958e-ab2f-4c89-8593-c0993d03a23b None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:13.951000 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3958e-ab2f-4c89-8593-c0993d03a23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:14.011779 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-74d3958e-ab2f-4c89-8593-c0993d03a23b None None] MFA Rules not processed for user `b5dbaf53464b44fe8d9e16f5013587b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:14.013209 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-74d3958e-ab2f-4c89-8593-c0993d03a23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:14.054451 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0a932d-0b98-424b-8390-f9a2cc81e788 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:14.061148 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0a932d-0b98-424b-8390-f9a2cc81e788 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:14.091277 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 741/1485] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:13 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:10:14.096101 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc2c92-3107-462f-bc95-2b74552701cb None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:14.097734 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc2c92-3107-462f-bc95-2b74552701cb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:14.098224 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc2c92-3107-462f-bc95-2b74552701cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:14.189799 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f7dc2c92-3107-462f-bc95-2b74552701cb None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:14.196894 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f7dc2c92-3107-462f-bc95-2b74552701cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:14.286539 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 745/1486] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:13 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:14.286539 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3958e-ab2f-4c89-8593-c0993d03a23b None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:14.295978 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-74d3958e-ab2f-4c89-8593-c0993d03a23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:14.350123 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3958e-ab2f-4c89-8593-c0993d03a23b None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:14.356104 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c3dcc-50e8-436e-9199-d9cfdc13335b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:14.361268 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c3dcc-50e8-436e-9199-d9cfdc13335b service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:14.361656 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c3dcc-50e8-436e-9199-d9cfdc13335b service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:14.362037 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c3dcc-50e8-436e-9199-d9cfdc13335b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:14.375984 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73c3dcc-50e8-436e-9199-d9cfdc13335b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:14.377523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73c3dcc-50e8-436e-9199-d9cfdc13335b service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:14.382583 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 742/1487] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:14.384576 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc2c92-3107-462f-bc95-2b74552701cb None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:14.386033 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f7dc2c92-3107-462f-bc95-2b74552701cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:14.410518 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 746/1488] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:10:14.418889 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d5280-4922-491f-ad8c-b3c079bd320e None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:14.419349 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d5280-4922-491f-ad8c-b3c079bd320e None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:14.419726 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d5280-4922-491f-ad8c-b3c079bd320e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:14.424135 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc2c92-3107-462f-bc95-2b74552701cb None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:14.447621 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0bd04-4248-4a01-b5d9-be02f2275cf7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:14.448328 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0bd04-4248-4a01-b5d9-be02f2275cf7 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:14.448627 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0bd04-4248-4a01-b5d9-be02f2275cf7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:14.449891 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0bd04-4248-4a01-b5d9-be02f2275cf7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:14.460374 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-edb0bd04-4248-4a01-b5d9-be02f2275cf7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:14.481472 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-b51d5280-4922-491f-ad8c-b3c079bd320e None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:14.506050 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-b51d5280-4922-491f-ad8c-b3c079bd320e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:14.557651 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 747/1489] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:14.560204 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d5280-4922-491f-ad8c-b3c079bd320e None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:14.607305 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d5280-4922-491f-ad8c-b3c079bd320e None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:14.637375 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca4e38-f5cc-4320-b60f-86d86ed53637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:14.639699 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca4e38-f5cc-4320-b60f-86d86ed53637 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:14.642572 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca4e38-f5cc-4320-b60f-86d86ed53637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:14.643609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca4e38-f5cc-4320-b60f-86d86ed53637 admin admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:14.647984 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:14.650440 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca4e38-f5cc-4320-b60f-86d86ed53637 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:10:14.650858 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca4e38-f5cc-4320-b60f-86d86ed53637 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:14.655366 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca4e38-f5cc-4320-b60f-86d86ed53637 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:14.723289 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 748/1490] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:10:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:10:14.726673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca4e38-f5cc-4320-b60f-86d86ed53637 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:14.770135 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0bd04-4248-4a01-b5d9-be02f2275cf7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:14.779582 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0bd04-4248-4a01-b5d9-be02f2275cf7 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:14.781095 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca4e38-f5cc-4320-b60f-86d86ed53637 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:14.818439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133f928-495a-4a42-9091-4570b6593167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:14.825080 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133f928-495a-4a42-9091-4570b6593167 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:14.827688 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133f928-495a-4a42-9091-4570b6593167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:14.829550 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133f928-495a-4a42-9091-4570b6593167 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:14.830658 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f133f928-495a-4a42-9091-4570b6593167 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:14.832829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 743/1491] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:14 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:10:14.841680 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f133f928-495a-4a42-9091-4570b6593167 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:14.904370 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:14.979560 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0bd04-4248-4a01-b5d9-be02f2275cf7 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:14.989354 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 749/1492] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:10:14 2022] POST /identity/v3/projects => generated 387 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:14.992732 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0bd04-4248-4a01-b5d9-be02f2275cf7 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:15.294540 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87898341-97ed-4fbf-a8a7-a7c3045f1732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:15.305598 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87898341-97ed-4fbf-a8a7-a7c3045f1732 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:15.306613 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87898341-97ed-4fbf-a8a7-a7c3045f1732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:15.307272 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87898341-97ed-4fbf-a8a7-a7c3045f1732 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:15.308128 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87898341-97ed-4fbf-a8a7-a7c3045f1732 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:15.314797 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87898341-97ed-4fbf-a8a7-a7c3045f1732 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:15.398594 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 744/1493] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:10:14 2022] POST /identity/v3/users => generated 294 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:15.404805 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133f928-495a-4a42-9091-4570b6593167 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:15.449964 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133f928-495a-4a42-9091-4570b6593167 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:15.460828 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a8cf2-1921-411f-b305-81d709473761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:15.464157 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a8cf2-1921-411f-b305-81d709473761 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:15.464806 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a8cf2-1921-411f-b305-81d709473761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:15.465279 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a8cf2-1921-411f-b305-81d709473761 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:15.466108 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a8cf2-1921-411f-b305-81d709473761 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:15.469259 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a8cf2-1921-411f-b305-81d709473761 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:15.500440 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 750/1494] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:15 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:15.504448 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87898341-97ed-4fbf-a8a7-a7c3045f1732 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:15.524364 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87898341-97ed-4fbf-a8a7-a7c3045f1732 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:15.554027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2da76b-1e03-4d20-b19a-88d5d7e966fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:15.556043 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2da76b-1e03-4d20-b19a-88d5d7e966fa admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:15.564722 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2da76b-1e03-4d20-b19a-88d5d7e966fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:15.566167 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2da76b-1e03-4d20-b19a-88d5d7e966fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:15.566892 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2da76b-1e03-4d20-b19a-88d5d7e966fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:15.572162 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2da76b-1e03-4d20-b19a-88d5d7e966fa admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:15.637030 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a8cf2-1921-411f-b305-81d709473761 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:15.639095 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 745/1495] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:15 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:10:15.662835 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a8cf2-1921-411f-b305-81d709473761 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:15.673876 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef52a8-04bc-4447-9f1a-3e8a2caae604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:15.681121 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef52a8-04bc-4447-9f1a-3e8a2caae604 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:15.685019 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef52a8-04bc-4447-9f1a-3e8a2caae604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:15.685583 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef52a8-04bc-4447-9f1a-3e8a2caae604 admin admin] PATH_INFO: `/v3/projects/6bf5b6a35c884d048bc06f7de25fce11/users/e2ad56b145af41969ddca8cf6de97c0c/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:15.768329 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef52a8-04bc-4447-9f1a-3e8a2caae604 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bf5b6a35c884d048bc06f7de25fce11, user_id=e2ad56b145af41969ddca8cf6de97c0c, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:15.783431 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef52a8-04bc-4447-9f1a-3e8a2caae604 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:15.832816 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b97aa-dbe7-4ce6-b420-76f7fad6e383 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:15.833027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 751/1496] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:10:15 2022] PUT /identity/v3/projects/6bf5b6a35c884d048bc06f7de25fce11/users/e2ad56b145af41969ddca8cf6de97c0c/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:10:15.834353 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b97aa-dbe7-4ce6-b420-76f7fad6e383 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:15.838323 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b97aa-dbe7-4ce6-b420-76f7fad6e383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:15.986917 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-9f1b97aa-dbe7-4ce6-b420-76f7fad6e383 None None] MFA Rules not processed for user `e2ad56b145af41969ddca8cf6de97c0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:15.998601 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9f1b97aa-dbe7-4ce6-b420-76f7fad6e383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:16.312388 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 746/1497] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 481 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:16.317038 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef52a8-04bc-4447-9f1a-3e8a2caae604 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:16.507887 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef52a8-04bc-4447-9f1a-3e8a2caae604 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:16.542720 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f740e43-f428-4354-9080-027b8d05edf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:16.543725 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f740e43-f428-4354-9080-027b8d05edf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:16.547187 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f740e43-f428-4354-9080-027b8d05edf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:16.548967 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f740e43-f428-4354-9080-027b8d05edf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:16.549779 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f740e43-f428-4354-9080-027b8d05edf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:16.560594 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f740e43-f428-4354-9080-027b8d05edf0 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:16.706753 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 752/1498] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:10:16 2022] POST /identity/v3/projects => generated 387 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:16.713043 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1b97aa-dbe7-4ce6-b420-76f7fad6e383 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:16.740855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1b97aa-dbe7-4ce6-b420-76f7fad6e383 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:17.008971 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:17.046125 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566e3a5-1b24-4b2f-8457-31a4fc22f974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:17.046758 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566e3a5-1b24-4b2f-8457-31a4fc22f974 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:17.047036 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566e3a5-1b24-4b2f-8457-31a4fc22f974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:17.048406 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566e3a5-1b24-4b2f-8457-31a4fc22f974 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:17.049242 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0566e3a5-1b24-4b2f-8457-31a4fc22f974 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:17.053403 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0566e3a5-1b24-4b2f-8457-31a4fc22f974 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:17.107455 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 747/1499] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:10:16 2022] POST /identity/v3/users => generated 300 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:17.112167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f740e43-f428-4354-9080-027b8d05edf0 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:17.131226 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f740e43-f428-4354-9080-027b8d05edf0 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:17.161555 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a544b4-3eac-4657-97ab-467f0abe2942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:17.162302 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a544b4-3eac-4657-97ab-467f0abe2942 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:17.162537 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a544b4-3eac-4657-97ab-467f0abe2942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:17.167381 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a544b4-3eac-4657-97ab-467f0abe2942 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:17.168561 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a544b4-3eac-4657-97ab-467f0abe2942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:17.174010 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a544b4-3eac-4657-97ab-467f0abe2942 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:17.226453 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 753/1500] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:17.227169 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566e3a5-1b24-4b2f-8457-31a4fc22f974 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:17.241389 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566e3a5-1b24-4b2f-8457-31a4fc22f974 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:17.261977 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a544b4-3eac-4657-97ab-467f0abe2942 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:17.262428 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32f4b3-628d-49ca-be97-fa3f1413a96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:17.263816 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-66a544b4-3eac-4657-97ab-467f0abe2942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:17.264721 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32f4b3-628d-49ca-be97-fa3f1413a96b admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:17.266061 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32f4b3-628d-49ca-be97-fa3f1413a96b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:17.266545 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32f4b3-628d-49ca-be97-fa3f1413a96b admin admin] PATH_INFO: `/v3/projects/3fa99f88c34c46e2949f9dd2eea82a38/users/4ab968cb4b604b8fb986ca5e7e4b7314/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:17.293061 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a544b4-3eac-4657-97ab-467f0abe2942 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:17.316751 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a9e16-ab89-47cd-a8d1-266a52b27a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:17.318504 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a9e16-ab89-47cd-a8d1-266a52b27a3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:17.318699 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a9e16-ab89-47cd-a8d1-266a52b27a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:17.318946 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a9e16-ab89-47cd-a8d1-266a52b27a3e admin admin] PATH_INFO: `/v3/users/b33a8b1f4b63461c9f78a046c1c41b77` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:17.335711 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a9e16-ab89-47cd-a8d1-266a52b27a3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b33a8b1f4b63461c9f78a046c1c41b77)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:17.338735 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a9e16-ab89-47cd-a8d1-266a52b27a3e admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:17.340533 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd32f4b3-628d-49ca-be97-fa3f1413a96b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fa99f88c34c46e2949f9dd2eea82a38, user_id=4ab968cb4b604b8fb986ca5e7e4b7314, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:17.345858 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd32f4b3-628d-49ca-be97-fa3f1413a96b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:17.366724 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 748/1501] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:10:17 2022] PUT /identity/v3/projects/3fa99f88c34c46e2949f9dd2eea82a38/users/4ab968cb4b604b8fb986ca5e7e4b7314/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:17.370466 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32f4b3-628d-49ca-be97-fa3f1413a96b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:17.534190 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-887a9e16-ab89-47cd-a8d1-266a52b27a3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b33a8b1f4b63461c9f78a046c1c41b77'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:17.562067 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32f4b3-628d-49ca-be97-fa3f1413a96b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:17.564854 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-887a9e16-ab89-47cd-a8d1-266a52b27a3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b33a8b1f4b63461c9f78a046c1c41b77'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:17.567500 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-887a9e16-ab89-47cd-a8d1-266a52b27a3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b33a8b1f4b63461c9f78a046c1c41b77'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:17.569475 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-887a9e16-ab89-47cd-a8d1-266a52b27a3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b33a8b1f4b63461c9f78a046c1c41b77'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:17.579360 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6512fb-9431-426e-a04d-1215c37e1d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:17.580536 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6512fb-9431-426e-a04d-1215c37e1d6d admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:17.581328 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6512fb-9431-426e-a04d-1215c37e1d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:17.583110 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6512fb-9431-426e-a04d-1215c37e1d6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:17.587108 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6512fb-9431-426e-a04d-1215c37e1d6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:17.591623 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6512fb-9431-426e-a04d-1215c37e1d6d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:17.601707 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-887a9e16-ab89-47cd-a8d1-266a52b27a3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b33a8b1f4b63461c9f78a046c1c41b77'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:17.623615 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:17.636068 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 749/1503] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:17.636780 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 754/1503] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:10:17 2022] DELETE /identity/v3/users/b33a8b1f4b63461c9f78a046c1c41b77 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:10:17.640179 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ba230-fea1-46ec-a99e-43ec568b9863 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:17.641122 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ba230-fea1-46ec-a99e-43ec568b9863 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:17.641649 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a9e16-ab89-47cd-a8d1-266a52b27a3e admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:17.641927 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ba230-fea1-46ec-a99e-43ec568b9863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:17.643777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-887a9e16-ab89-47cd-a8d1-266a52b27a3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:17.694154 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-5c3ba230-fea1-46ec-a99e-43ec568b9863 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:17.695509 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5c3ba230-fea1-46ec-a99e-43ec568b9863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:17.801444 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a9e16-ab89-47cd-a8d1-266a52b27a3e admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:17.810323 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 750/1504] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:17.819831 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e3530-cad3-42e8-bed1-253355ed8608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:17.821007 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e3530-cad3-42e8-bed1-253355ed8608 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:17.821662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e3530-cad3-42e8-bed1-253355ed8608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:17.822961 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e3530-cad3-42e8-bed1-253355ed8608 admin admin] PATH_INFO: `/v3/projects/3fa99f88c34c46e2949f9dd2eea82a38/users/4ab968cb4b604b8fb986ca5e7e4b7314/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:17.830659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e3530-cad3-42e8-bed1-253355ed8608 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fa99f88c34c46e2949f9dd2eea82a38, user_id=4ab968cb4b604b8fb986ca5e7e4b7314, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:17.854629 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e3530-cad3-42e8-bed1-253355ed8608 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:17.880022 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 755/1505] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:10:17 2022] PUT /identity/v3/projects/3fa99f88c34c46e2949f9dd2eea82a38/users/4ab968cb4b604b8fb986ca5e7e4b7314/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:17.892265 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc04ddf-f6af-48f5-bb18-118ff323769b None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:17.892776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc04ddf-f6af-48f5-bb18-118ff323769b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:17.893111 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc04ddf-f6af-48f5-bb18-118ff323769b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:18.007865 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-9cc04ddf-f6af-48f5-bb18-118ff323769b None None] MFA Rules not processed for user `4ab968cb4b604b8fb986ca5e7e4b7314`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:18.015387 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9cc04ddf-f6af-48f5-bb18-118ff323769b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:18.032239 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e3530-cad3-42e8-bed1-253355ed8608 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:18.037631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-520e3530-cad3-42e8-bed1-253355ed8608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:18.135800 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:18.263441 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e3530-cad3-42e8-bed1-253355ed8608 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:18.269369 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2ad14-ec7a-437f-8925-d7a08bcd3071 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:18.270417 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2ad14-ec7a-437f-8925-d7a08bcd3071 service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:18.272080 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2ad14-ec7a-437f-8925-d7a08bcd3071 service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:18.272559 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2ad14-ec7a-437f-8925-d7a08bcd3071 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:18.318847 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 751/1506] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:17 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 428 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:18.319840 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc04ddf-f6af-48f5-bb18-118ff323769b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:18.331843 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9cc04ddf-f6af-48f5-bb18-118ff323769b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:18.403212 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e2ad14-ec7a-437f-8925-d7a08bcd3071 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:18.408398 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e2ad14-ec7a-437f-8925-d7a08bcd3071 service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:18.438940 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 756/1507] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:10:18.632217 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc04ddf-f6af-48f5-bb18-118ff323769b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:18.643144 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3630496b-3cfb-4715-b8d1-0498ac2db298 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:18.644098 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630496b-3cfb-4715-b8d1-0498ac2db298 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:18.647096 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630496b-3cfb-4715-b8d1-0498ac2db298 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:18.647601 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630496b-3cfb-4715-b8d1-0498ac2db298 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:18.671177 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3630496b-3cfb-4715-b8d1-0498ac2db298 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:18.673095 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3630496b-3cfb-4715-b8d1-0498ac2db298 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:18.701399 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 752/1508] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:18 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:10:18.780496 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2ad14-ec7a-437f-8925-d7a08bcd3071 service neutron] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:18.820670 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2ad14-ec7a-437f-8925-d7a08bcd3071 service neutron] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:18.836126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ba7d0-383b-4159-bc34-d6cb97f4903f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:18.838209 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ba7d0-383b-4159-bc34-d6cb97f4903f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:18.842028 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ba7d0-383b-4159-bc34-d6cb97f4903f admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:18.842378 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ba7d0-383b-4159-bc34-d6cb97f4903f admin admin] PATH_INFO: `/v3/users/e2ad56b145af41969ddca8cf6de97c0c` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:18.847644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ba7d0-383b-4159-bc34-d6cb97f4903f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2ad56b145af41969ddca8cf6de97c0c)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:18.854022 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75ba7d0-383b-4159-bc34-d6cb97f4903f admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:18.994669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:19.036760 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-e75ba7d0-383b-4159-bc34-d6cb97f4903f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2ad56b145af41969ddca8cf6de97c0c'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:19.087334 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-e75ba7d0-383b-4159-bc34-d6cb97f4903f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2ad56b145af41969ddca8cf6de97c0c'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:19.089033 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-e75ba7d0-383b-4159-bc34-d6cb97f4903f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2ad56b145af41969ddca8cf6de97c0c'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:19.091476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-e75ba7d0-383b-4159-bc34-d6cb97f4903f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2ad56b145af41969ddca8cf6de97c0c'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:19.115931 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a15151-9c37-48dd-be92-94226522ce3f None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:19.116223 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a15151-9c37-48dd-be92-94226522ce3f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:19.117764 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-e75ba7d0-383b-4159-bc34-d6cb97f4903f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2ad56b145af41969ddca8cf6de97c0c'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:19.118552 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a15151-9c37-48dd-be92-94226522ce3f None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:19.120504 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 753/1509] 10.222.0.23 () {54 vars in 921 bytes} [Thu Oct 13 13:10:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:10:19.201236 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 757/1510] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:10:18 2022] DELETE /identity/v3/users/e2ad56b145af41969ddca8cf6de97c0c => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:19.206614 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d714b6-22fd-4616-ae2a-8654d3c1769f None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:19.206901 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d714b6-22fd-4616-ae2a-8654d3c1769f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:19.207503 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d714b6-22fd-4616-ae2a-8654d3c1769f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:19.247299 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-56d714b6-22fd-4616-ae2a-8654d3c1769f None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:19.250096 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-56d714b6-22fd-4616-ae2a-8654d3c1769f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:19.567282 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 754/1511] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:19.668189 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ba7d0-383b-4159-bc34-d6cb97f4903f admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:19.680512 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-e75ba7d0-383b-4159-bc34-d6cb97f4903f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:19.819154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75ba7d0-383b-4159-bc34-d6cb97f4903f admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:19.828140 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61323f7-609f-4536-8e8d-e27f8d69dbc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:19.829280 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61323f7-609f-4536-8e8d-e27f8d69dbc3 service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:19.829934 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61323f7-609f-4536-8e8d-e27f8d69dbc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:19.830398 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61323f7-609f-4536-8e8d-e27f8d69dbc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:19.857210 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61323f7-609f-4536-8e8d-e27f8d69dbc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:19.861864 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61323f7-609f-4536-8e8d-e27f8d69dbc3 service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:19.881921 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 758/1512] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:10:19.981349 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ff2c2-f957-4561-9d0d-563daafe4fab None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:19.983671 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ff2c2-f957-4561-9d0d-563daafe4fab None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:19.984061 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ff2c2-f957-4561-9d0d-563daafe4fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:20.046638 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6c7ff2c2-f957-4561-9d0d-563daafe4fab None None] MFA Rules not processed for user `12700d985a724595af630c81e63f86d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:20.050608 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6c7ff2c2-f957-4561-9d0d-563daafe4fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:20.266099 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 755/1513] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Oct 13 13:10:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:21.044608 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61323f7-609f-4536-8e8d-e27f8d69dbc3 service neutron] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:21.046144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-d61323f7-609f-4536-8e8d-e27f8d69dbc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:21.090116 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61323f7-609f-4536-8e8d-e27f8d69dbc3 service neutron] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:21.119910 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb022512-aa6b-4c6c-9967-3427a0f59b36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:21.120861 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb022512-aa6b-4c6c-9967-3427a0f59b36 service nova] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:21.121769 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb022512-aa6b-4c6c-9967-3427a0f59b36 service nova] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:21.124455 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb022512-aa6b-4c6c-9967-3427a0f59b36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:21.161146 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb022512-aa6b-4c6c-9967-3427a0f59b36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:21.167241 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb022512-aa6b-4c6c-9967-3427a0f59b36 service nova] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:21.210707 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 759/1514] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:10:22.041396 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ff2c2-f957-4561-9d0d-563daafe4fab None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:22.045025 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6c7ff2c2-f957-4561-9d0d-563daafe4fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:22.071666 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ff2c2-f957-4561-9d0d-563daafe4fab None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:22.084205 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35288abe-80c7-4118-981d-0ad515965643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:22.084953 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35288abe-80c7-4118-981d-0ad515965643 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:22.085522 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35288abe-80c7-4118-981d-0ad515965643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:22.085916 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35288abe-80c7-4118-981d-0ad515965643 admin admin] PATH_INFO: `/v3/projects/6bf5b6a35c884d048bc06f7de25fce11` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:22.088987 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35288abe-80c7-4118-981d-0ad515965643 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bf5b6a35c884d048bc06f7de25fce11)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:22.092796 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35288abe-80c7-4118-981d-0ad515965643 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:22.110172 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:22.211034 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-35288abe-80c7-4118-981d-0ad515965643 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bf5b6a35c884d048bc06f7de25fce11', 'request_id': 'req-35288abe-80c7-4118-981d-0ad515965643'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:22.223532 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-35288abe-80c7-4118-981d-0ad515965643 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bf5b6a35c884d048bc06f7de25fce11', 'request_id': 'req-35288abe-80c7-4118-981d-0ad515965643'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:22.242096 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-35288abe-80c7-4118-981d-0ad515965643 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bf5b6a35c884d048bc06f7de25fce11', 'request_id': 'req-35288abe-80c7-4118-981d-0ad515965643'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:22.268708 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-35288abe-80c7-4118-981d-0ad515965643 admin admin] The token cache is being invalidate because project 6bf5b6a35c884d048bc06f7de25fce11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:10:22.269294 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-35288abe-80c7-4118-981d-0ad515965643 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:22.275823 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-35288abe-80c7-4118-981d-0ad515965643 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:22.282022 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 756/1515] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:10:22 2022] DELETE /identity/v3/projects/6bf5b6a35c884d048bc06f7de25fce11 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:22.283734 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb022512-aa6b-4c6c-9967-3427a0f59b36 service nova] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:22.298654 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb022512-aa6b-4c6c-9967-3427a0f59b36 service nova] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:22.381399 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3fc6d-3d5b-4dec-9ca1-cffc87921dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:22.382453 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3fc6d-3d5b-4dec-9ca1-cffc87921dea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:22.383094 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3fc6d-3d5b-4dec-9ca1-cffc87921dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:22.383609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3fc6d-3d5b-4dec-9ca1-cffc87921dea admin admin] PATH_INFO: `/v3/users/4ab968cb4b604b8fb986ca5e7e4b7314` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:22.385100 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e3fc6d-3d5b-4dec-9ca1-cffc87921dea admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ab968cb4b604b8fb986ca5e7e4b7314)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:22.387741 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e3fc6d-3d5b-4dec-9ca1-cffc87921dea admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:22.488011 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-41e3fc6d-3d5b-4dec-9ca1-cffc87921dea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ab968cb4b604b8fb986ca5e7e4b7314'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:22.489446 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:22.504391 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-41e3fc6d-3d5b-4dec-9ca1-cffc87921dea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ab968cb4b604b8fb986ca5e7e4b7314'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:22.507215 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-41e3fc6d-3d5b-4dec-9ca1-cffc87921dea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ab968cb4b604b8fb986ca5e7e4b7314'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:22.508087 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-41e3fc6d-3d5b-4dec-9ca1-cffc87921dea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ab968cb4b604b8fb986ca5e7e4b7314'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:22.520686 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-41e3fc6d-3d5b-4dec-9ca1-cffc87921dea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ab968cb4b604b8fb986ca5e7e4b7314'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:22.552471 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 760/1516] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:10:22 2022] DELETE /identity/v3/users/4ab968cb4b604b8fb986ca5e7e4b7314 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:22.703172 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7aa0da-aa35-43bf-a488-c3bd00fec5cf None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:22.703552 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7aa0da-aa35-43bf-a488-c3bd00fec5cf None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:22.703900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7aa0da-aa35-43bf-a488-c3bd00fec5cf None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:22.705424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 757/1517] 10.222.0.23 () {54 vars in 921 bytes} [Thu Oct 13 13:10:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 13 13:10:22.730804 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b4814-af29-43dd-906b-3088ec84a3fc None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:22.731168 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b4814-af29-43dd-906b-3088ec84a3fc None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:22.731386 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b4814-af29-43dd-906b-3088ec84a3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:22.793814 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-059b4814-af29-43dd-906b-3088ec84a3fc None None] MFA Rules not processed for user `12700d985a724595af630c81e63f86d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:22.799748 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-059b4814-af29-43dd-906b-3088ec84a3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:22.994326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7aa0da-aa35-43bf-a488-c3bd00fec5cf None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:22.997346 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3a7aa0da-aa35-43bf-a488-c3bd00fec5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:23.028196 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 761/1518] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Oct 13 13:10:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:23.163841 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7aa0da-aa35-43bf-a488-c3bd00fec5cf None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:23.176049 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439a1d8-e7fd-4090-86d7-ce1f7cef172d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:23.176850 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439a1d8-e7fd-4090-86d7-ce1f7cef172d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:23.177500 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439a1d8-e7fd-4090-86d7-ce1f7cef172d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:23.178031 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439a1d8-e7fd-4090-86d7-ce1f7cef172d admin admin] PATH_INFO: `/v3/projects/7e54a1d888f74dd683f52403d6f93fce` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:23.179324 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b439a1d8-e7fd-4090-86d7-ce1f7cef172d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e54a1d888f74dd683f52403d6f93fce)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:23.182027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b439a1d8-e7fd-4090-86d7-ce1f7cef172d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:23.313319 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b439a1d8-e7fd-4090-86d7-ce1f7cef172d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e54a1d888f74dd683f52403d6f93fce', 'request_id': 'req-b439a1d8-e7fd-4090-86d7-ce1f7cef172d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:23.333786 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b439a1d8-e7fd-4090-86d7-ce1f7cef172d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e54a1d888f74dd683f52403d6f93fce', 'request_id': 'req-b439a1d8-e7fd-4090-86d7-ce1f7cef172d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:23.376382 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b439a1d8-e7fd-4090-86d7-ce1f7cef172d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e54a1d888f74dd683f52403d6f93fce', 'request_id': 'req-b439a1d8-e7fd-4090-86d7-ce1f7cef172d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:23.416151 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b439a1d8-e7fd-4090-86d7-ce1f7cef172d admin admin] The token cache is being invalidate because project 7e54a1d888f74dd683f52403d6f93fce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:10:23.416661 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b439a1d8-e7fd-4090-86d7-ce1f7cef172d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:23.417624 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b439a1d8-e7fd-4090-86d7-ce1f7cef172d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:23.420561 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 758/1519] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:10:22 2022] DELETE /identity/v3/projects/7e54a1d888f74dd683f52403d6f93fce => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:23.423715 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059b4814-af29-43dd-906b-3088ec84a3fc None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:23.451370 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059b4814-af29-43dd-906b-3088ec84a3fc None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:23.458791 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439a1d8-e7fd-4090-86d7-ce1f7cef172d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:23.460739 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b439a1d8-e7fd-4090-86d7-ce1f7cef172d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:23.558363 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:23.645684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e521c100-7fca-4b56-88ac-b5dfbe7ce941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:23.646740 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521c100-7fca-4b56-88ac-b5dfbe7ce941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:23.647294 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521c100-7fca-4b56-88ac-b5dfbe7ce941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:23.647768 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521c100-7fca-4b56-88ac-b5dfbe7ce941 admin admin] PATH_INFO: `/v3/users/b4462e9ad1b0467498edd185c4c20557` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:23.649668 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e521c100-7fca-4b56-88ac-b5dfbe7ce941 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4462e9ad1b0467498edd185c4c20557)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:23.651990 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e521c100-7fca-4b56-88ac-b5dfbe7ce941 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:23.671176 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439a1d8-e7fd-4090-86d7-ce1f7cef172d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:23.685279 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04ab63-93d1-44af-b847-3ea0731a27c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:23.686031 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04ab63-93d1-44af-b847-3ea0731a27c7 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:23.686233 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04ab63-93d1-44af-b847-3ea0731a27c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:23.686459 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04ab63-93d1-44af-b847-3ea0731a27c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:23.691679 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6b04ab63-93d1-44af-b847-3ea0731a27c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:23.714662 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b04ab63-93d1-44af-b847-3ea0731a27c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:23.718865 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b04ab63-93d1-44af-b847-3ea0731a27c7 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:23.748890 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 759/1520] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:10:23.807498 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-e521c100-7fca-4b56-88ac-b5dfbe7ce941 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4462e9ad1b0467498edd185c4c20557'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:23.855739 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-e521c100-7fca-4b56-88ac-b5dfbe7ce941 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4462e9ad1b0467498edd185c4c20557'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:23.856548 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-e521c100-7fca-4b56-88ac-b5dfbe7ce941 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4462e9ad1b0467498edd185c4c20557'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:23.857193 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-e521c100-7fca-4b56-88ac-b5dfbe7ce941 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4462e9ad1b0467498edd185c4c20557'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:23.866748 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:23.887150 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-e521c100-7fca-4b56-88ac-b5dfbe7ce941 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4462e9ad1b0467498edd185c4c20557'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:23.927638 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 762/1521] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:10:23 2022] DELETE /identity/v3/users/b4462e9ad1b0467498edd185c4c20557 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:25.290367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04ab63-93d1-44af-b847-3ea0731a27c7 service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:25.292685 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6b04ab63-93d1-44af-b847-3ea0731a27c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:25.317868 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04ab63-93d1-44af-b847-3ea0731a27c7 service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:25.412261 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b747850c-21e1-43d2-a709-3428a10e8db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:25.413113 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747850c-21e1-43d2-a709-3428a10e8db7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:25.413560 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747850c-21e1-43d2-a709-3428a10e8db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:25.414047 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747850c-21e1-43d2-a709-3428a10e8db7 admin admin] PATH_INFO: `/v3/projects/15acee4862cc4fdfab62c492de2c55b2` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:25.415579 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b747850c-21e1-43d2-a709-3428a10e8db7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15acee4862cc4fdfab62c492de2c55b2)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:25.418563 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b747850c-21e1-43d2-a709-3428a10e8db7 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:25.520033 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b747850c-21e1-43d2-a709-3428a10e8db7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15acee4862cc4fdfab62c492de2c55b2', 'request_id': 'req-b747850c-21e1-43d2-a709-3428a10e8db7'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:25.529615 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b747850c-21e1-43d2-a709-3428a10e8db7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15acee4862cc4fdfab62c492de2c55b2', 'request_id': 'req-b747850c-21e1-43d2-a709-3428a10e8db7'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:25.537283 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b747850c-21e1-43d2-a709-3428a10e8db7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15acee4862cc4fdfab62c492de2c55b2', 'request_id': 'req-b747850c-21e1-43d2-a709-3428a10e8db7'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:25.555049 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b747850c-21e1-43d2-a709-3428a10e8db7 admin admin] The token cache is being invalidate because project 15acee4862cc4fdfab62c492de2c55b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:10:25.555491 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b747850c-21e1-43d2-a709-3428a10e8db7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:25.556212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b747850c-21e1-43d2-a709-3428a10e8db7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:25.558159 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 760/1522] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:10:25 2022] DELETE /identity/v3/projects/15acee4862cc4fdfab62c492de2c55b2 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:25.564543 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29960f3e-7c79-42b3-9607-4aa466a9e5be None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:25.565630 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29960f3e-7c79-42b3-9607-4aa466a9e5be None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:25.566283 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29960f3e-7c79-42b3-9607-4aa466a9e5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:25.589010 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-29960f3e-7c79-42b3-9607-4aa466a9e5be None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:25.590559 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-29960f3e-7c79-42b3-9607-4aa466a9e5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:25.701558 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 763/1523] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:25.702829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb2d87-3785-498a-8d2e-ad0bc18b81f3 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:25.703139 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb2d87-3785-498a-8d2e-ad0bc18b81f3 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:25.703443 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb2d87-3785-498a-8d2e-ad0bc18b81f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:25.726730 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c2eb2d87-3785-498a-8d2e-ad0bc18b81f3 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:25.729406 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c2eb2d87-3785-498a-8d2e-ad0bc18b81f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:25.755011 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 761/1524] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:10:25.757499 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29960f3e-7c79-42b3-9607-4aa466a9e5be None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:25.773888 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29960f3e-7c79-42b3-9607-4aa466a9e5be None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:25.780367 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d81a2-2385-4d41-b613-d1858901d17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:25.780884 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb2d87-3785-498a-8d2e-ad0bc18b81f3 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:25.781129 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d81a2-2385-4d41-b613-d1858901d17e admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:25.782155 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d81a2-2385-4d41-b613-d1858901d17e admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:25.782383 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c2eb2d87-3785-498a-8d2e-ad0bc18b81f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:25.782553 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d81a2-2385-4d41-b613-d1858901d17e admin admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:25.783851 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54d81a2-2385-4d41-b613-d1858901d17e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:10:25.783851 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54d81a2-2385-4d41-b613-d1858901d17e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:25.785537 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54d81a2-2385-4d41-b613-d1858901d17e admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:25.833953 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb2d87-3785-498a-8d2e-ad0bc18b81f3 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:25.859317 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 764/1525] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:10:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:10:25.859317 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d81a2-2385-4d41-b613-d1858901d17e admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:25.882983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d81a2-2385-4d41-b613-d1858901d17e admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:25.888736 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdfb7ce-8d21-4ed1-929e-e6458297fe17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:25.890501 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdfb7ce-8d21-4ed1-929e-e6458297fe17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:25.890501 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdfb7ce-8d21-4ed1-929e-e6458297fe17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:25.890652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdfb7ce-8d21-4ed1-929e-e6458297fe17 admin admin] PATH_INFO: `/v3/projects/3fa99f88c34c46e2949f9dd2eea82a38` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:25.892034 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdfb7ce-8d21-4ed1-929e-e6458297fe17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fa99f88c34c46e2949f9dd2eea82a38)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:25.893935 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdfb7ce-8d21-4ed1-929e-e6458297fe17 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:25.898330 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97825fa8-0f8d-4e73-8250-e21887c9a00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:25.899511 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97825fa8-0f8d-4e73-8250-e21887c9a00c admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:25.899797 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97825fa8-0f8d-4e73-8250-e21887c9a00c admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:25.902874 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97825fa8-0f8d-4e73-8250-e21887c9a00c admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:25.903587 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97825fa8-0f8d-4e73-8250-e21887c9a00c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:25.907651 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97825fa8-0f8d-4e73-8250-e21887c9a00c admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:25.974140 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 765/1526] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:10:25 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:25.980091 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97825fa8-0f8d-4e73-8250-e21887c9a00c admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:25.981901 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:26.032910 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7bdfb7ce-8d21-4ed1-929e-e6458297fe17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fa99f88c34c46e2949f9dd2eea82a38', 'request_id': 'req-7bdfb7ce-8d21-4ed1-929e-e6458297fe17'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:26.057134 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7bdfb7ce-8d21-4ed1-929e-e6458297fe17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fa99f88c34c46e2949f9dd2eea82a38', 'request_id': 'req-7bdfb7ce-8d21-4ed1-929e-e6458297fe17'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:26.064973 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7bdfb7ce-8d21-4ed1-929e-e6458297fe17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fa99f88c34c46e2949f9dd2eea82a38', 'request_id': 'req-7bdfb7ce-8d21-4ed1-929e-e6458297fe17'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:26.087362 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7bdfb7ce-8d21-4ed1-929e-e6458297fe17 admin admin] The token cache is being invalidate because project 3fa99f88c34c46e2949f9dd2eea82a38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:10:26.087657 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7bdfb7ce-8d21-4ed1-929e-e6458297fe17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:26.088531 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7bdfb7ce-8d21-4ed1-929e-e6458297fe17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:26.090813 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 762/1527] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:10:25 2022] DELETE /identity/v3/projects/3fa99f88c34c46e2949f9dd2eea82a38 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:26.095370 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a49c7-17f8-4190-9f75-631a54ceaf96 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:26.095701 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a49c7-17f8-4190-9f75-631a54ceaf96 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:26.095977 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a49c7-17f8-4190-9f75-631a54ceaf96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:26.100650 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97825fa8-0f8d-4e73-8250-e21887c9a00c admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:26.122370 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-cc2a49c7-17f8-4190-9f75-631a54ceaf96 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:26.123628 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-cc2a49c7-17f8-4190-9f75-631a54ceaf96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:26.229520 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:26.257667 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 763/1528] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:26.266644 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5514e2-cbd2-48fc-8b59-282ca6d3e3d5 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:26.267015 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5514e2-cbd2-48fc-8b59-282ca6d3e3d5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:26.267263 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5514e2-cbd2-48fc-8b59-282ca6d3e3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:26.277629 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781d8aa-4b5b-449d-9d74-9d2e956e3b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:26.280416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781d8aa-4b5b-449d-9d74-9d2e956e3b26 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:26.281143 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781d8aa-4b5b-449d-9d74-9d2e956e3b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:26.281558 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781d8aa-4b5b-449d-9d74-9d2e956e3b26 admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:26.283531 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4781d8aa-4b5b-449d-9d74-9d2e956e3b26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:26.292349 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4781d8aa-4b5b-449d-9d74-9d2e956e3b26 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:26.308353 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-4c5514e2-cbd2-48fc-8b59-282ca6d3e3d5 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:26.311445 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4c5514e2-cbd2-48fc-8b59-282ca6d3e3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:26.329699 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 764/1529] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:10:26.334976 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5514e2-cbd2-48fc-8b59-282ca6d3e3d5 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:26.352923 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 766/1530] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:10:25 2022] POST /identity/v3/users => generated 287 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:10:26.354665 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781d8aa-4b5b-449d-9d74-9d2e956e3b26 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:26.357117 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-4781d8aa-4b5b-449d-9d74-9d2e956e3b26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:26.362706 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5514e2-cbd2-48fc-8b59-282ca6d3e3d5 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:26.375873 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a4cd6-c8f8-40be-ab19-1ad1dbe41a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:26.376836 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a4cd6-c8f8-40be-ab19-1ad1dbe41a1b admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:26.376990 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a4cd6-c8f8-40be-ab19-1ad1dbe41a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:26.377397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a4cd6-c8f8-40be-ab19-1ad1dbe41a1b admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:26.378124 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a4cd6-c8f8-40be-ab19-1ad1dbe41a1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:10:26.379949 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a4cd6-c8f8-40be-ab19-1ad1dbe41a1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:26.382728 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a4cd6-c8f8-40be-ab19-1ad1dbe41a1b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:26.395731 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781d8aa-4b5b-449d-9d74-9d2e956e3b26 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:26.403851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 765/1531] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:10:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:10:26.410603 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a4cd6-c8f8-40be-ab19-1ad1dbe41a1b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:26.412521 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d417fa7-b8c0-4c0d-86f6-fc9e4c96ec39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:26.413160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d417fa7-b8c0-4c0d-86f6-fc9e4c96ec39 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:26.413439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d417fa7-b8c0-4c0d-86f6-fc9e4c96ec39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:26.413710 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d417fa7-b8c0-4c0d-86f6-fc9e4c96ec39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:26.414366 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d417fa7-b8c0-4c0d-86f6-fc9e4c96ec39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:26.416068 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d417fa7-b8c0-4c0d-86f6-fc9e4c96ec39 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:26.425080 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a4cd6-c8f8-40be-ab19-1ad1dbe41a1b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:26.434459 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1f00d-73d3-468d-98fd-92307a73bc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:26.435813 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1f00d-73d3-468d-98fd-92307a73bc79 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:26.436461 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1f00d-73d3-468d-98fd-92307a73bc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:26.436936 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1f00d-73d3-468d-98fd-92307a73bc79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:26.438068 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c1f00d-73d3-468d-98fd-92307a73bc79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:26.442884 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 767/1532] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:26.443911 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d417fa7-b8c0-4c0d-86f6-fc9e4c96ec39 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:26.444162 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c1f00d-73d3-468d-98fd-92307a73bc79 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:26.463007 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d417fa7-b8c0-4c0d-86f6-fc9e4c96ec39 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:26.471594 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32ee74-fc0a-4bf6-823d-c2ba2824bb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:26.472308 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32ee74-fc0a-4bf6-823d-c2ba2824bb54 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:26.473042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32ee74-fc0a-4bf6-823d-c2ba2824bb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:26.473042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba32ee74-fc0a-4bf6-823d-c2ba2824bb54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:26.473996 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba32ee74-fc0a-4bf6-823d-c2ba2824bb54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:26.475655 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba32ee74-fc0a-4bf6-823d-c2ba2824bb54 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:26.492045 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 768/1533] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:26.492211 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 766/1534] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:10:26 2022] POST /identity/v3/projects => generated 379 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:10:26.496280 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32ee74-fc0a-4bf6-823d-c2ba2824bb54 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:26.496424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1f00d-73d3-468d-98fd-92307a73bc79 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:26.510608 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba32ee74-fc0a-4bf6-823d-c2ba2824bb54 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:26.600470 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1f00d-73d3-468d-98fd-92307a73bc79 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:26.607461 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d3b9c-4318-4957-92bf-1b8a32f542c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:26.608393 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d3b9c-4318-4957-92bf-1b8a32f542c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:26.608751 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d3b9c-4318-4957-92bf-1b8a32f542c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:26.609032 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d3b9c-4318-4957-92bf-1b8a32f542c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:26.609977 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d3b9c-4318-4957-92bf-1b8a32f542c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:26.612262 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d3b9c-4318-4957-92bf-1b8a32f542c7 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:26.616605 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8d931-b42f-415e-9ed4-8499639647b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:26.618572 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8d931-b42f-415e-9ed4-8499639647b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:26.619077 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8d931-b42f-415e-9ed4-8499639647b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:26.619560 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8d931-b42f-415e-9ed4-8499639647b9 admin admin] PATH_INFO: `/v3/projects/a0873e26915f4efeb27ac6308f48d628/users/6874c9dfe0fb435fbe2ede442ac9d0db/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:26.642490 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 767/1535] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:10:26 2022] POST /identity/v3/users => generated 290 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:26.648255 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d3b9c-4318-4957-92bf-1b8a32f542c7 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:26.665247 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8d931-b42f-415e-9ed4-8499639647b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0873e26915f4efeb27ac6308f48d628, user_id=6874c9dfe0fb435fbe2ede442ac9d0db, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:26.672821 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8d931-b42f-415e-9ed4-8499639647b9 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:26.678394 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:26.691129 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d3b9c-4318-4957-92bf-1b8a32f542c7 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:26.711325 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 769/1536] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:10:26 2022] PUT /identity/v3/projects/a0873e26915f4efeb27ac6308f48d628/users/6874c9dfe0fb435fbe2ede442ac9d0db/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:26.719900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa447f5-c18a-4ddd-bdbe-95e772011c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:26.719900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa447f5-c18a-4ddd-bdbe-95e772011c7b admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:26.719900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa447f5-c18a-4ddd-bdbe-95e772011c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:26.719900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa447f5-c18a-4ddd-bdbe-95e772011c7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:26.720328 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa447f5-c18a-4ddd-bdbe-95e772011c7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:26.722850 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b0c11-a385-4fd2-b53a-45d3f337b441 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:26.723357 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b0c11-a385-4fd2-b53a-45d3f337b441 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:26.723826 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b0c11-a385-4fd2-b53a-45d3f337b441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:26.724453 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa447f5-c18a-4ddd-bdbe-95e772011c7b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:26.769715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 768/1537] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:26.770877 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa447f5-c18a-4ddd-bdbe-95e772011c7b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:26.783812 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-c10b0c11-a385-4fd2-b53a-45d3f337b441 None None] MFA Rules not processed for user `6874c9dfe0fb435fbe2ede442ac9d0db`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:26.785054 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-c10b0c11-a385-4fd2-b53a-45d3f337b441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:26.819379 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa447f5-c18a-4ddd-bdbe-95e772011c7b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:26.917939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 770/1538] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:26.937948 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b0c11-a385-4fd2-b53a-45d3f337b441 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:26.939325 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f84a5fe-4e16-4a8b-8a62-3edb04a180b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:26.940263 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-c10b0c11-a385-4fd2-b53a-45d3f337b441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:26.940750 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84a5fe-4e16-4a8b-8a62-3edb04a180b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:26.941033 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84a5fe-4e16-4a8b-8a62-3edb04a180b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:26.941304 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84a5fe-4e16-4a8b-8a62-3edb04a180b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:26.941960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f84a5fe-4e16-4a8b-8a62-3edb04a180b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:26.944165 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f84a5fe-4e16-4a8b-8a62-3edb04a180b8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:26.959752 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 769/1539] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:26.961304 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f84a5fe-4e16-4a8b-8a62-3edb04a180b8 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:26.979817 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:27.003687 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f84a5fe-4e16-4a8b-8a62-3edb04a180b8 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:27.010711 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b0c11-a385-4fd2-b53a-45d3f337b441 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:27.012819 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674a29d-b8d6-4ae3-be24-a31979251130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:27.013727 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674a29d-b8d6-4ae3-be24-a31979251130 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:27.014095 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674a29d-b8d6-4ae3-be24-a31979251130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:27.014648 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674a29d-b8d6-4ae3-be24-a31979251130 admin admin] PATH_INFO: `/v3/projects/f9a60ab0ddf94adbad30380c602df80d/users/62a47982dc12444cbe6363d49341a539/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:27.020101 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45a47f-be8f-4727-855f-f68eb96d38d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:27.021346 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45a47f-be8f-4727-855f-f68eb96d38d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:27.021651 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45a47f-be8f-4727-855f-f68eb96d38d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:27.021923 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45a47f-be8f-4727-855f-f68eb96d38d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:27.032000 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f45a47f-be8f-4727-855f-f68eb96d38d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:27.033444 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f45a47f-be8f-4727-855f-f68eb96d38d1 service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:27.034778 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c674a29d-b8d6-4ae3-be24-a31979251130 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9a60ab0ddf94adbad30380c602df80d, user_id=62a47982dc12444cbe6363d49341a539, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:27.038226 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c674a29d-b8d6-4ae3-be24-a31979251130 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:27.045190 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 771/1540] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 13 13:10:27.056798 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 770/1541] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:10:26 2022] PUT /identity/v3/projects/f9a60ab0ddf94adbad30380c602df80d/users/62a47982dc12444cbe6363d49341a539/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:27.067450 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe115b7-2cd7-406f-bbcf-9ba561566af6 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:27.067625 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe115b7-2cd7-406f-bbcf-9ba561566af6 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:27.067907 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe115b7-2cd7-406f-bbcf-9ba561566af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:27.079021 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674a29d-b8d6-4ae3-be24-a31979251130 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:27.101452 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-ffe115b7-2cd7-406f-bbcf-9ba561566af6 None None] MFA Rules not processed for user `62a47982dc12444cbe6363d49341a539`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:27.103416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-ffe115b7-2cd7-406f-bbcf-9ba561566af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:27.170450 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674a29d-b8d6-4ae3-be24-a31979251130 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:27.175633 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c541d1-5577-4143-9128-c5b4748bd228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:27.176338 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c541d1-5577-4143-9128-c5b4748bd228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:27.176733 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c541d1-5577-4143-9128-c5b4748bd228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:27.177032 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c541d1-5577-4143-9128-c5b4748bd228 admin admin] PATH_INFO: `/v3/users/6874c9dfe0fb435fbe2ede442ac9d0db` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:27.178561 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c541d1-5577-4143-9128-c5b4748bd228 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6874c9dfe0fb435fbe2ede442ac9d0db)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:27.180603 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c541d1-5577-4143-9128-c5b4748bd228 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:27.199397 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 772/1542] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:27.200342 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe115b7-2cd7-406f-bbcf-9ba561566af6 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:27.215272 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe115b7-2cd7-406f-bbcf-9ba561566af6 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:27.220248 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c537f26-6ada-4589-8300-3dc7bc45162b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:27.220863 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c537f26-6ada-4589-8300-3dc7bc45162b admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:27.221143 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c537f26-6ada-4589-8300-3dc7bc45162b admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:27.221413 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c537f26-6ada-4589-8300-3dc7bc45162b admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:27.222067 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c537f26-6ada-4589-8300-3dc7bc45162b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:27.224306 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c537f26-6ada-4589-8300-3dc7bc45162b admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:27.262224 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 773/1543] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:10:27 2022] POST /identity/v3/projects => generated 381 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:10:27.263576 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c537f26-6ada-4589-8300-3dc7bc45162b admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:27.288738 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-05c541d1-5577-4143-9128-c5b4748bd228 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6874c9dfe0fb435fbe2ede442ac9d0db'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:27.302048 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-05c541d1-5577-4143-9128-c5b4748bd228 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6874c9dfe0fb435fbe2ede442ac9d0db'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:27.310543 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-05c541d1-5577-4143-9128-c5b4748bd228 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6874c9dfe0fb435fbe2ede442ac9d0db'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:27.321221 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-05c541d1-5577-4143-9128-c5b4748bd228 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6874c9dfe0fb435fbe2ede442ac9d0db'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:27.326965 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:27.337819 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-05c541d1-5577-4143-9128-c5b4748bd228 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6874c9dfe0fb435fbe2ede442ac9d0db'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:27.342122 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c537f26-6ada-4589-8300-3dc7bc45162b admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:27.346884 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc93cc-2c55-4113-8af8-e39d94b3881a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:27.347667 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc93cc-2c55-4113-8af8-e39d94b3881a admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:27.347950 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc93cc-2c55-4113-8af8-e39d94b3881a admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:27.348222 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc93cc-2c55-4113-8af8-e39d94b3881a admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:27.348857 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc93cc-2c55-4113-8af8-e39d94b3881a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:27.350779 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc93cc-2c55-4113-8af8-e39d94b3881a admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:27.357055 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 771/1544] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:10:27 2022] DELETE /identity/v3/users/6874c9dfe0fb435fbe2ede442ac9d0db => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:27.361901 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e108d-73d2-4fdd-aa85-0ea9ef0047f4 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:27.362183 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e108d-73d2-4fdd-aa85-0ea9ef0047f4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:27.362457 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e108d-73d2-4fdd-aa85-0ea9ef0047f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:27.378463 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 774/1545] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:10:27 2022] POST /identity/v3/users => generated 297 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:27.379884 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc93cc-2c55-4113-8af8-e39d94b3881a admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:27.381589 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-13dc93cc-2c55-4113-8af8-e39d94b3881a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:27.382629 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-193e108d-73d2-4fdd-aa85-0ea9ef0047f4 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:27.383860 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-193e108d-73d2-4fdd-aa85-0ea9ef0047f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:27.396016 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc93cc-2c55-4113-8af8-e39d94b3881a admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:27.430775 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:27.461909 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 772/1546] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:27.469610 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e2568-aaaf-4f2a-9cbd-945cd528c726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:27.470322 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e2568-aaaf-4f2a-9cbd-945cd528c726 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:27.470500 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e2568-aaaf-4f2a-9cbd-945cd528c726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:27.470772 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e2568-aaaf-4f2a-9cbd-945cd528c726 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:27.471462 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e2568-aaaf-4f2a-9cbd-945cd528c726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:27.472200 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e108d-73d2-4fdd-aa85-0ea9ef0047f4 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:27.473486 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e2568-aaaf-4f2a-9cbd-945cd528c726 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:27.473950 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-193e108d-73d2-4fdd-aa85-0ea9ef0047f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:27.487280 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 775/1547] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:27 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:10:27.491334 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e2568-aaaf-4f2a-9cbd-945cd528c726 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:27.504600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e2568-aaaf-4f2a-9cbd-945cd528c726 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:27.509983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643e73f-92a0-4525-927e-53c04a89da20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:27.510585 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643e73f-92a0-4525-927e-53c04a89da20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:27.511091 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643e73f-92a0-4525-927e-53c04a89da20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:27.511296 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643e73f-92a0-4525-927e-53c04a89da20 admin admin] PATH_INFO: `/v3/projects/2a53a8e06c674b21be97af21d5ae7602/users/654f176b29bb4a2088212b1a68275472/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:27.532653 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643e73f-92a0-4525-927e-53c04a89da20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a53a8e06c674b21be97af21d5ae7602, user_id=654f176b29bb4a2088212b1a68275472, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:27.537074 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643e73f-92a0-4525-927e-53c04a89da20 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:27.558503 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 776/1548] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:10:27 2022] PUT /identity/v3/projects/2a53a8e06c674b21be97af21d5ae7602/users/654f176b29bb4a2088212b1a68275472/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:10:27.559887 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643e73f-92a0-4525-927e-53c04a89da20 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:27.561914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e108d-73d2-4fdd-aa85-0ea9ef0047f4 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:27.627637 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15238d-0fcd-4adb-93d1-cbf587c2a50e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:27.628874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15238d-0fcd-4adb-93d1-cbf587c2a50e service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:27.629342 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15238d-0fcd-4adb-93d1-cbf587c2a50e service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:27.629613 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15238d-0fcd-4adb-93d1-cbf587c2a50e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:27.658963 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643e73f-92a0-4525-927e-53c04a89da20 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:27.665546 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a36a9-1005-4b83-85ce-2c5eed6fb18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:27.667096 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a36a9-1005-4b83-85ce-2c5eed6fb18e admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:27.667917 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a36a9-1005-4b83-85ce-2c5eed6fb18e admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:27.668382 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a36a9-1005-4b83-85ce-2c5eed6fb18e admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:27.670107 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a36a9-1005-4b83-85ce-2c5eed6fb18e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:27.673146 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a36a9-1005-4b83-85ce-2c5eed6fb18e admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:27.688679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 777/1549] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:27 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:27.693700 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a36a9-1005-4b83-85ce-2c5eed6fb18e admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:27.698605 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:27.712921 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c15238d-0fcd-4adb-93d1-cbf587c2a50e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:27.716220 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c15238d-0fcd-4adb-93d1-cbf587c2a50e service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:27.725444 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a36a9-1005-4b83-85ce-2c5eed6fb18e admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:27.728543 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 773/1550] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 13 13:10:27.733896 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a2896c-ad73-4303-8b47-79402e1a698e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:27.734874 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2896c-ad73-4303-8b47-79402e1a698e admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:27.735337 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2896c-ad73-4303-8b47-79402e1a698e admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:27.735759 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2896c-ad73-4303-8b47-79402e1a698e admin admin] PATH_INFO: `/v3/projects/2a53a8e06c674b21be97af21d5ae7602/users/654f176b29bb4a2088212b1a68275472/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:27.737967 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a2896c-ad73-4303-8b47-79402e1a698e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a53a8e06c674b21be97af21d5ae7602, user_id=654f176b29bb4a2088212b1a68275472, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:27.740809 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a2896c-ad73-4303-8b47-79402e1a698e admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:27.754332 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 778/1551] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:10:27 2022] PUT /identity/v3/projects/2a53a8e06c674b21be97af21d5ae7602/users/654f176b29bb4a2088212b1a68275472/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:27.767709 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342646cf-21b5-4157-83a8-8cdec8de6a01 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:27.768123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342646cf-21b5-4157-83a8-8cdec8de6a01 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:27.768459 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342646cf-21b5-4157-83a8-8cdec8de6a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:27.804372 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-342646cf-21b5-4157-83a8-8cdec8de6a01 None None] MFA Rules not processed for user `654f176b29bb4a2088212b1a68275472`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:27.805669 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-342646cf-21b5-4157-83a8-8cdec8de6a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:27.915402 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 774/1552] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:27 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:10:27.920558 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a2896c-ad73-4303-8b47-79402e1a698e admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:27.922616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-03a2896c-ad73-4303-8b47-79402e1a698e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:27.992256 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a2896c-ad73-4303-8b47-79402e1a698e admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:27.996299 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e991b0ff-f312-4095-9f5c-8854734feccd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:27.997288 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991b0ff-f312-4095-9f5c-8854734feccd service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:27.997381 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991b0ff-f312-4095-9f5c-8854734feccd service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:27.997738 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991b0ff-f312-4095-9f5c-8854734feccd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:28.046871 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e991b0ff-f312-4095-9f5c-8854734feccd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:28.050453 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e991b0ff-f312-4095-9f5c-8854734feccd service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:28.077114 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 779/1553] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:27 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:10:28.589813 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342646cf-21b5-4157-83a8-8cdec8de6a01 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:28.600458 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342646cf-21b5-4157-83a8-8cdec8de6a01 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:28.605870 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44294be-bfc0-499f-8864-ab3a141d7bc7 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654f176b29bb4a2088212b1a68275472', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a53a8e06c674b21be97af21d5ae7602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:28.606624 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44294be-bfc0-499f-8864-ab3a141d7bc7 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:28.606916 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44294be-bfc0-499f-8864-ab3a141d7bc7 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:28.607223 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44294be-bfc0-499f-8864-ab3a141d7bc7 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:28.607915 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44294be-bfc0-499f-8864-ab3a141d7bc7 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:10:28.608205 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44294be-bfc0-499f-8864-ab3a141d7bc7 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:28.609795 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44294be-bfc0-499f-8864-ab3a141d7bc7 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:28.622623 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 775/1554] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:10:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:10:28.626576 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e991b0ff-f312-4095-9f5c-8854734feccd service cinder] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:28.638964 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e991b0ff-f312-4095-9f5c-8854734feccd service cinder] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:28.661796 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc5af2-0344-45b6-a492-21a7b454de09 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654f176b29bb4a2088212b1a68275472', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a53a8e06c674b21be97af21d5ae7602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:28.662544 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc5af2-0344-45b6-a492-21a7b454de09 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:28.662924 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc5af2-0344-45b6-a492-21a7b454de09 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:28.663307 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc5af2-0344-45b6-a492-21a7b454de09 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:28.664149 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc5af2-0344-45b6-a492-21a7b454de09 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:28.666331 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc5af2-0344-45b6-a492-21a7b454de09 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:28.697348 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 780/1555] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:10:28 2022] POST /identity/v3/projects => generated 353 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:28.702360 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44294be-bfc0-499f-8864-ab3a141d7bc7 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:28.719611 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44294be-bfc0-499f-8864-ab3a141d7bc7 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:28.784063 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc5af2-0344-45b6-a492-21a7b454de09 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:28.786059 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-45dc5af2-0344-45b6-a492-21a7b454de09 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:28.793510 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:28.812807 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee64f3-591e-4b68-a530-24293db250e2 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654f176b29bb4a2088212b1a68275472', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a53a8e06c674b21be97af21d5ae7602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:28.814174 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee64f3-591e-4b68-a530-24293db250e2 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:28.814591 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee64f3-591e-4b68-a530-24293db250e2 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:28.814963 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee64f3-591e-4b68-a530-24293db250e2 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:28.815912 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee64f3-591e-4b68-a530-24293db250e2 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:10:28.816852 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee64f3-591e-4b68-a530-24293db250e2 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:28.818957 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee64f3-591e-4b68-a530-24293db250e2 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:28.840009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 776/1556] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:10:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 13:10:28.931092 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee64f3-591e-4b68-a530-24293db250e2 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:28.945113 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee64f3-591e-4b68-a530-24293db250e2 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:28.948244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc5af2-0344-45b6-a492-21a7b454de09 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:28.952854 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48622181-0014-4d5f-9437-cd3e75e3ca03 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '654f176b29bb4a2088212b1a68275472', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a53a8e06c674b21be97af21d5ae7602', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:28.954651 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fce195-bddf-4cb2-af7c-a491aacf2892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:28.955280 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48622181-0014-4d5f-9437-cd3e75e3ca03 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:28.955521 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fce195-bddf-4cb2-af7c-a491aacf2892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:28.955898 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fce195-bddf-4cb2-af7c-a491aacf2892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:28.956397 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fce195-bddf-4cb2-af7c-a491aacf2892 admin admin] PATH_INFO: `/v3/projects/a0873e26915f4efeb27ac6308f48d628` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:28.956610 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48622181-0014-4d5f-9437-cd3e75e3ca03 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:28.957026 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48622181-0014-4d5f-9437-cd3e75e3ca03 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] PATH_INFO: `/v3/projects/b612e3b632c94cff8a58bc091b3180b3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:28.958596 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fce195-bddf-4cb2-af7c-a491aacf2892 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0873e26915f4efeb27ac6308f48d628)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:28.959402 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48622181-0014-4d5f-9437-cd3e75e3ca03 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b612e3b632c94cff8a58bc091b3180b3)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:28.964026 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48622181-0014-4d5f-9437-cd3e75e3ca03 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:28.964428 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fce195-bddf-4cb2-af7c-a491aacf2892 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:29.062769 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-02fce195-bddf-4cb2-af7c-a491aacf2892 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0873e26915f4efeb27ac6308f48d628', 'request_id': 'req-02fce195-bddf-4cb2-af7c-a491aacf2892'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:29.086264 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-48622181-0014-4d5f-9437-cd3e75e3ca03 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b612e3b632c94cff8a58bc091b3180b3', 'request_id': 'req-48622181-0014-4d5f-9437-cd3e75e3ca03'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:29.087197 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-02fce195-bddf-4cb2-af7c-a491aacf2892 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0873e26915f4efeb27ac6308f48d628', 'request_id': 'req-02fce195-bddf-4cb2-af7c-a491aacf2892'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:29.099540 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:29.101455 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-02fce195-bddf-4cb2-af7c-a491aacf2892 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0873e26915f4efeb27ac6308f48d628', 'request_id': 'req-02fce195-bddf-4cb2-af7c-a491aacf2892'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:29.110330 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-48622181-0014-4d5f-9437-cd3e75e3ca03 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b612e3b632c94cff8a58bc091b3180b3', 'request_id': 'req-48622181-0014-4d5f-9437-cd3e75e3ca03'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:29.126105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-48622181-0014-4d5f-9437-cd3e75e3ca03 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b612e3b632c94cff8a58bc091b3180b3', 'request_id': 'req-48622181-0014-4d5f-9437-cd3e75e3ca03'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:29.127407 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-02fce195-bddf-4cb2-af7c-a491aacf2892 admin admin] The token cache is being invalidate because project a0873e26915f4efeb27ac6308f48d628 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:10:29.127952 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-02fce195-bddf-4cb2-af7c-a491aacf2892 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:29.129441 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-02fce195-bddf-4cb2-af7c-a491aacf2892 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:29.132051 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 781/1557] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:10:28 2022] DELETE /identity/v3/projects/a0873e26915f4efeb27ac6308f48d628 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:29.142320 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab85336-b486-4eed-86fa-696f48181535 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:29.143163 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab85336-b486-4eed-86fa-696f48181535 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:29.144775 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab85336-b486-4eed-86fa-696f48181535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:29.161259 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-48622181-0014-4d5f-9437-cd3e75e3ca03 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] The token cache is being invalidate because project b612e3b632c94cff8a58bc091b3180b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:10:29.162330 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-48622181-0014-4d5f-9437-cd3e75e3ca03 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:29.168976 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-48622181-0014-4d5f-9437-cd3e75e3ca03 tempest-VolumeQuotaClassesTest-2005457919 tempest-VolumeQuotaClassesTest-2005457919-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:29.174071 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 777/1558] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:10:28 2022] DELETE /identity/v3/projects/b612e3b632c94cff8a58bc091b3180b3 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:29.216616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-5ab85336-b486-4eed-86fa-696f48181535 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:29.218313 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-5ab85336-b486-4eed-86fa-696f48181535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:29.322012 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 782/1559] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:29.322100 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d23d13-adbb-49fd-a26b-22a6ad2ffe70 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:29.322100 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d23d13-adbb-49fd-a26b-22a6ad2ffe70 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:29.322100 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d23d13-adbb-49fd-a26b-22a6ad2ffe70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:29.378750 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f0d23d13-adbb-49fd-a26b-22a6ad2ffe70 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:29.379565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f0d23d13-adbb-49fd-a26b-22a6ad2ffe70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:29.407422 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 778/1560] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:29.407963 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab85336-b486-4eed-86fa-696f48181535 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:29.432319 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab85336-b486-4eed-86fa-696f48181535 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:29.449111 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa9239-2e78-4774-ab20-fe0682eee7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:29.450162 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa9239-2e78-4774-ab20-fe0682eee7b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:29.453529 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa9239-2e78-4774-ab20-fe0682eee7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:29.454287 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa9239-2e78-4774-ab20-fe0682eee7b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:29.458232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa9239-2e78-4774-ab20-fe0682eee7b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:10:29.459866 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa9239-2e78-4774-ab20-fe0682eee7b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:29.467853 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa9239-2e78-4774-ab20-fe0682eee7b5 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:29.502450 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 783/1561] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:10:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 13:10:29.518928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d23d13-adbb-49fd-a26b-22a6ad2ffe70 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:29.535590 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d23d13-adbb-49fd-a26b-22a6ad2ffe70 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:29.553769 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad8207-9da0-4bbe-960f-56717812fa0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:29.554645 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad8207-9da0-4bbe-960f-56717812fa0d admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:29.555641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad8207-9da0-4bbe-960f-56717812fa0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:29.556057 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad8207-9da0-4bbe-960f-56717812fa0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:29.556851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ad8207-9da0-4bbe-960f-56717812fa0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:29.558926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ad8207-9da0-4bbe-960f-56717812fa0d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:29.616778 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 779/1562] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:10:29 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:29.621443 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa9239-2e78-4774-ab20-fe0682eee7b5 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:29.636603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa9239-2e78-4774-ab20-fe0682eee7b5 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:29.676439 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad8207-9da0-4bbe-960f-56717812fa0d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:29.678797 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-47ad8207-9da0-4bbe-960f-56717812fa0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:29.733476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:29.852869 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b099e-966d-4644-b4f2-b5da8b0dfdfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:29.853835 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b099e-966d-4644-b4f2-b5da8b0dfdfa admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:29.854285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b099e-966d-4644-b4f2-b5da8b0dfdfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:29.854701 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b099e-966d-4644-b4f2-b5da8b0dfdfa admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:29.855701 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b099e-966d-4644-b4f2-b5da8b0dfdfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:29.858521 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b099e-966d-4644-b4f2-b5da8b0dfdfa admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:29.876943 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad8207-9da0-4bbe-960f-56717812fa0d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:29.884553 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937143a2-b8bf-4a01-ade3-2df6f9ecd373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:29.885660 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937143a2-b8bf-4a01-ade3-2df6f9ecd373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:29.886505 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937143a2-b8bf-4a01-ade3-2df6f9ecd373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:29.886944 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937143a2-b8bf-4a01-ade3-2df6f9ecd373 admin admin] PATH_INFO: `/v3/users/62a47982dc12444cbe6363d49341a539` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:29.889136 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937143a2-b8bf-4a01-ade3-2df6f9ecd373 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62a47982dc12444cbe6363d49341a539)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:29.891773 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937143a2-b8bf-4a01-ade3-2df6f9ecd373 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:29.892161 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 784/1563] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:10:29 2022] POST /identity/v3/users => generated 287 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:29.896375 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b099e-966d-4644-b4f2-b5da8b0dfdfa admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:29.919886 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b099e-966d-4644-b4f2-b5da8b0dfdfa admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:29.938064 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947ddef-272a-4ea4-9fd8-fca88de575f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:29.939095 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947ddef-272a-4ea4-9fd8-fca88de575f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:29.939537 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947ddef-272a-4ea4-9fd8-fca88de575f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:29.939952 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947ddef-272a-4ea4-9fd8-fca88de575f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:29.940729 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5947ddef-272a-4ea4-9fd8-fca88de575f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:29.942742 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5947ddef-272a-4ea4-9fd8-fca88de575f1 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:29.973097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-937143a2-b8bf-4a01-ade3-2df6f9ecd373 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62a47982dc12444cbe6363d49341a539'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:29.977730 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 785/1564] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:29.983416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947ddef-272a-4ea4-9fd8-fca88de575f1 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:29.994018 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-937143a2-b8bf-4a01-ade3-2df6f9ecd373 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62a47982dc12444cbe6363d49341a539'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:29.995530 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-937143a2-b8bf-4a01-ade3-2df6f9ecd373 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62a47982dc12444cbe6363d49341a539'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:30.009894 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947ddef-272a-4ea4-9fd8-fca88de575f1 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:30.012835 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-937143a2-b8bf-4a01-ade3-2df6f9ecd373 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62a47982dc12444cbe6363d49341a539'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:30.026779 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8947622-fa09-471d-97f4-a75f0764e88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:30.028235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8947622-fa09-471d-97f4-a75f0764e88e admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:30.028512 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8947622-fa09-471d-97f4-a75f0764e88e admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:30.030704 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8947622-fa09-471d-97f4-a75f0764e88e admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:30.031435 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8947622-fa09-471d-97f4-a75f0764e88e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:30.033063 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8947622-fa09-471d-97f4-a75f0764e88e admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:30.038049 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-937143a2-b8bf-4a01-ade3-2df6f9ecd373 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62a47982dc12444cbe6363d49341a539'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:30.047307 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 780/1565] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:10:29 2022] DELETE /identity/v3/users/62a47982dc12444cbe6363d49341a539 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:30.053207 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7e649-c86b-497e-96d3-370dd1ef8496 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:30.053597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7e649-c86b-497e-96d3-370dd1ef8496 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:30.054048 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7e649-c86b-497e-96d3-370dd1ef8496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:30.058865 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 786/1566] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:30.063642 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8947622-fa09-471d-97f4-a75f0764e88e admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:30.065600 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-c8947622-fa09-471d-97f4-a75f0764e88e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:30.086875 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-35f7e649-c86b-497e-96d3-370dd1ef8496 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:30.088315 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-35f7e649-c86b-497e-96d3-370dd1ef8496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:30.111491 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8947622-fa09-471d-97f4-a75f0764e88e admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:30.159839 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:30.240365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9d51c-12d5-4f80-aa21-8f67d1ed304d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:30.241420 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9d51c-12d5-4f80-aa21-8f67d1ed304d admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:30.241880 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9d51c-12d5-4f80-aa21-8f67d1ed304d admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:30.242319 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9d51c-12d5-4f80-aa21-8f67d1ed304d admin admin] PATH_INFO: `/v3/projects/238de2aafc0048f3b538d4624293d6bb/users/e08e29ebce1c4559acad7b3111802070/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:30.245889 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 781/1567] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:30.268608 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe9d51c-12d5-4f80-aa21-8f67d1ed304d admin admin] RBAC: Authorizing `identity:create_grant(project_id=238de2aafc0048f3b538d4624293d6bb, user_id=e08e29ebce1c4559acad7b3111802070, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:30.271706 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe9d51c-12d5-4f80-aa21-8f67d1ed304d admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:30.297052 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 787/1568] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:10:30 2022] PUT /identity/v3/projects/238de2aafc0048f3b538d4624293d6bb/users/e08e29ebce1c4559acad7b3111802070/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:10:30.305655 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f7e649-c86b-497e-96d3-370dd1ef8496 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:30.308021 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-35f7e649-c86b-497e-96d3-370dd1ef8496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:30.311567 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef725c09-cec2-4db5-9317-4bdeaf4135dd None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:30.311822 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef725c09-cec2-4db5-9317-4bdeaf4135dd None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:30.312053 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef725c09-cec2-4db5-9317-4bdeaf4135dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:30.468965 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-ef725c09-cec2-4db5-9317-4bdeaf4135dd None None] MFA Rules not processed for user `e08e29ebce1c4559acad7b3111802070`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:30.470180 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-ef725c09-cec2-4db5-9317-4bdeaf4135dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:30.499461 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f7e649-c86b-497e-96d3-370dd1ef8496 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:30.563310 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0664cf-aab9-47c7-aad3-154467d2c8df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:30.564090 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0664cf-aab9-47c7-aad3-154467d2c8df service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:30.564389 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0664cf-aab9-47c7-aad3-154467d2c8df service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:30.564711 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0664cf-aab9-47c7-aad3-154467d2c8df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:30.615681 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 788/1569] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:30.627925 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef725c09-cec2-4db5-9317-4bdeaf4135dd None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:30.630116 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-ef725c09-cec2-4db5-9317-4bdeaf4135dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:30.634977 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:30.646656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0664cf-aab9-47c7-aad3-154467d2c8df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:30.647940 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0664cf-aab9-47c7-aad3-154467d2c8df service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:30.661395 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 782/1570] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:10:30.731955 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef725c09-cec2-4db5-9317-4bdeaf4135dd None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:30.736679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c84579-7e0b-478d-bf6d-9cd5cc931172 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ce234c5b4643b2ac6341e0928f3830', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:30.737351 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c84579-7e0b-478d-bf6d-9cd5cc931172 service glance] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:30.737690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c84579-7e0b-478d-bf6d-9cd5cc931172 service glance] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:30.737840 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c84579-7e0b-478d-bf6d-9cd5cc931172 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:30.747815 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c84579-7e0b-478d-bf6d-9cd5cc931172 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:30.749740 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c84579-7e0b-478d-bf6d-9cd5cc931172 service glance] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:30.761018 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 789/1571] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 13 13:10:31.027200 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0664cf-aab9-47c7-aad3-154467d2c8df service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:31.028913 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-bb0664cf-aab9-47c7-aad3-154467d2c8df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:31.049703 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0664cf-aab9-47c7-aad3-154467d2c8df service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:31.061626 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d63e60-9fe3-4b84-abd5-6c3c8e202de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:31.062452 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d63e60-9fe3-4b84-abd5-6c3c8e202de1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:31.062731 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d63e60-9fe3-4b84-abd5-6c3c8e202de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:31.063003 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d63e60-9fe3-4b84-abd5-6c3c8e202de1 admin admin] PATH_INFO: `/v3/projects/f9a60ab0ddf94adbad30380c602df80d` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:31.064935 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d63e60-9fe3-4b84-abd5-6c3c8e202de1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9a60ab0ddf94adbad30380c602df80d)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:31.068521 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d63e60-9fe3-4b84-abd5-6c3c8e202de1 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:31.185210 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:31.227387 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-80d63e60-9fe3-4b84-abd5-6c3c8e202de1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9a60ab0ddf94adbad30380c602df80d', 'request_id': 'req-80d63e60-9fe3-4b84-abd5-6c3c8e202de1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:31.251292 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-80d63e60-9fe3-4b84-abd5-6c3c8e202de1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9a60ab0ddf94adbad30380c602df80d', 'request_id': 'req-80d63e60-9fe3-4b84-abd5-6c3c8e202de1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:31.266091 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-80d63e60-9fe3-4b84-abd5-6c3c8e202de1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9a60ab0ddf94adbad30380c602df80d', 'request_id': 'req-80d63e60-9fe3-4b84-abd5-6c3c8e202de1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:31.290882 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-80d63e60-9fe3-4b84-abd5-6c3c8e202de1 admin admin] The token cache is being invalidate because project f9a60ab0ddf94adbad30380c602df80d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:10:31.291370 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-80d63e60-9fe3-4b84-abd5-6c3c8e202de1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:31.293551 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-80d63e60-9fe3-4b84-abd5-6c3c8e202de1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:31.300735 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 783/1572] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:10:31 2022] DELETE /identity/v3/projects/f9a60ab0ddf94adbad30380c602df80d => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:31.301929 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c84579-7e0b-478d-bf6d-9cd5cc931172 service glance] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:31.324582 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c84579-7e0b-478d-bf6d-9cd5cc931172 service glance] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:31.523428 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e763d-f51f-4b07-b103-8b67fdfc3912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:31.537070 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e763d-f51f-4b07-b103-8b67fdfc3912 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:31.538847 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e763d-f51f-4b07-b103-8b67fdfc3912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:31.539860 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e763d-f51f-4b07-b103-8b67fdfc3912 admin admin] PATH_INFO: `/v3/users/654f176b29bb4a2088212b1a68275472` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:31.548146 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e763d-f51f-4b07-b103-8b67fdfc3912 admin admin] RBAC: Authorizing `identity:delete_user(user_id=654f176b29bb4a2088212b1a68275472)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:31.554789 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e763d-f51f-4b07-b103-8b67fdfc3912 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:31.665905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-8f8e763d-f51f-4b07-b103-8b67fdfc3912 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '654f176b29bb4a2088212b1a68275472'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:31.682186 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-8f8e763d-f51f-4b07-b103-8b67fdfc3912 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '654f176b29bb4a2088212b1a68275472'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:31.688477 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-8f8e763d-f51f-4b07-b103-8b67fdfc3912 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '654f176b29bb4a2088212b1a68275472'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:31.690861 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-8f8e763d-f51f-4b07-b103-8b67fdfc3912 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '654f176b29bb4a2088212b1a68275472'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:31.715077 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-8f8e763d-f51f-4b07-b103-8b67fdfc3912 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '654f176b29bb4a2088212b1a68275472'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:31.776293 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 790/1573] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:10:31 2022] DELETE /identity/v3/users/654f176b29bb4a2088212b1a68275472 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:32.124312 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb4148-15bd-4af1-ae3b-1b3f3b49c2dd None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:32.126004 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb4148-15bd-4af1-ae3b-1b3f3b49c2dd None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:32.126760 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb4148-15bd-4af1-ae3b-1b3f3b49c2dd None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:32.128789 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 784/1574] 10.222.0.23 () {54 vars in 921 bytes} [Thu Oct 13 13:10:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:10:32.144579 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364f81f-0702-46f6-9239-a1c1ad118bff None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:32.144956 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364f81f-0702-46f6-9239-a1c1ad118bff None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:32.145527 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364f81f-0702-46f6-9239-a1c1ad118bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:32.159357 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:32.193742 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-c364f81f-0702-46f6-9239-a1c1ad118bff None None] MFA Rules not processed for user `12700d985a724595af630c81e63f86d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:32.195026 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-c364f81f-0702-46f6-9239-a1c1ad118bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:32.271162 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb4148-15bd-4af1-ae3b-1b3f3b49c2dd None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:32.273190 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-59cb4148-15bd-4af1-ae3b-1b3f3b49c2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:32.360104 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 791/1575] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Oct 13 13:10:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:10:32.376167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c364f81f-0702-46f6-9239-a1c1ad118bff None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:32.378533 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-c364f81f-0702-46f6-9239-a1c1ad118bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:32.396979 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c364f81f-0702-46f6-9239-a1c1ad118bff None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:32.401431 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb4148-15bd-4af1-ae3b-1b3f3b49c2dd None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:32.408915 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9604d4d-40e6-4f25-a5e2-f7fc98d79843 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:32.409298 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c0d09-3da5-45fc-81ac-03a07cf45aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:32.410771 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c0d09-3da5-45fc-81ac-03a07cf45aa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:32.411247 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c0d09-3da5-45fc-81ac-03a07cf45aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:32.411956 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9604d4d-40e6-4f25-a5e2-f7fc98d79843 service nova] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:32.412086 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c0d09-3da5-45fc-81ac-03a07cf45aa8 admin admin] PATH_INFO: `/v3/projects/2a53a8e06c674b21be97af21d5ae7602` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:32.412417 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9604d4d-40e6-4f25-a5e2-f7fc98d79843 service nova] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:32.412792 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9604d4d-40e6-4f25-a5e2-f7fc98d79843 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:32.420228 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583c0d09-3da5-45fc-81ac-03a07cf45aa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a53a8e06c674b21be97af21d5ae7602)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:32.422458 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583c0d09-3da5-45fc-81ac-03a07cf45aa8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:32.430830 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9604d4d-40e6-4f25-a5e2-f7fc98d79843 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:32.432627 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9604d4d-40e6-4f25-a5e2-f7fc98d79843 service nova] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:32.444292 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 792/1576] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:10:32.493801 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-583c0d09-3da5-45fc-81ac-03a07cf45aa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a53a8e06c674b21be97af21d5ae7602', 'request_id': 'req-583c0d09-3da5-45fc-81ac-03a07cf45aa8'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:32.503522 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-583c0d09-3da5-45fc-81ac-03a07cf45aa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a53a8e06c674b21be97af21d5ae7602', 'request_id': 'req-583c0d09-3da5-45fc-81ac-03a07cf45aa8'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:32.511953 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-583c0d09-3da5-45fc-81ac-03a07cf45aa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a53a8e06c674b21be97af21d5ae7602', 'request_id': 'req-583c0d09-3da5-45fc-81ac-03a07cf45aa8'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:32.529131 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-583c0d09-3da5-45fc-81ac-03a07cf45aa8 admin admin] The token cache is being invalidate because project 2a53a8e06c674b21be97af21d5ae7602 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:10:32.529424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-583c0d09-3da5-45fc-81ac-03a07cf45aa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:32.530092 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-583c0d09-3da5-45fc-81ac-03a07cf45aa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:32.531655 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 785/1577] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:10:32 2022] DELETE /identity/v3/projects/2a53a8e06c674b21be97af21d5ae7602 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:32.536407 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13169f-3441-4ca0-8952-7a0e5329ee7c None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:32.536791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13169f-3441-4ca0-8952-7a0e5329ee7c None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:32.537126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13169f-3441-4ca0-8952-7a0e5329ee7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:32.560159 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-3e13169f-3441-4ca0-8952-7a0e5329ee7c None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:32.561384 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-3e13169f-3441-4ca0-8952-7a0e5329ee7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:32.628365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 793/1578] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:32.638407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622ebbf-4547-4876-ac58-8360961b97d9 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:32.638691 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622ebbf-4547-4876-ac58-8360961b97d9 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:32.638969 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622ebbf-4547-4876-ac58-8360961b97d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:32.659237 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8622ebbf-4547-4876-ac58-8360961b97d9 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:32.660719 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8622ebbf-4547-4876-ac58-8360961b97d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:32.668809 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 786/1579] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:32.674560 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13169f-3441-4ca0-8952-7a0e5329ee7c None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:32.685650 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13169f-3441-4ca0-8952-7a0e5329ee7c None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:32.690344 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb0248-f7bd-44c5-8e2b-fb19ae075409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:32.690950 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb0248-f7bd-44c5-8e2b-fb19ae075409 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:32.691257 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb0248-f7bd-44c5-8e2b-fb19ae075409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:32.691527 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb0248-f7bd-44c5-8e2b-fb19ae075409 admin admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:32.692175 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb0248-f7bd-44c5-8e2b-fb19ae075409 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:10:32.692473 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb0248-f7bd-44c5-8e2b-fb19ae075409 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:32.694094 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb0248-f7bd-44c5-8e2b-fb19ae075409 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:32.707101 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 794/1580] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:10:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:10:32.711612 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622ebbf-4547-4876-ac58-8360961b97d9 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:32.721821 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622ebbf-4547-4876-ac58-8360961b97d9 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:32.726556 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639887ab-7632-48d7-ac0d-a7d0b8a470bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:32.727221 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639887ab-7632-48d7-ac0d-a7d0b8a470bb admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:32.727505 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639887ab-7632-48d7-ac0d-a7d0b8a470bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:32.727814 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639887ab-7632-48d7-ac0d-a7d0b8a470bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:32.728452 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639887ab-7632-48d7-ac0d-a7d0b8a470bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:32.730360 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639887ab-7632-48d7-ac0d-a7d0b8a470bb admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:32.770376 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 787/1581] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:10:32 2022] POST /identity/v3/projects => generated 365 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:32.774583 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb0248-f7bd-44c5-8e2b-fb19ae075409 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:32.785063 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb0248-f7bd-44c5-8e2b-fb19ae075409 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:32.859291 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c38197-7673-4a2d-b368-2f4c96e4e0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:32.860206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c38197-7673-4a2d-b368-2f4c96e4e0a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:32.860657 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c38197-7673-4a2d-b368-2f4c96e4e0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:32.861254 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c38197-7673-4a2d-b368-2f4c96e4e0a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:32.861927 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c38197-7673-4a2d-b368-2f4c96e4e0a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:32.864015 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c38197-7673-4a2d-b368-2f4c96e4e0a3 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:32.884794 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:32.892034 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 795/1582] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:10:32 2022] POST /identity/v3/users => generated 283 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:32.898977 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639887ab-7632-48d7-ac0d-a7d0b8a470bb admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:32.915026 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639887ab-7632-48d7-ac0d-a7d0b8a470bb admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:32.933472 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3341e4-aa51-4ec9-8b9d-56a0d14c1f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:32.934107 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3341e4-aa51-4ec9-8b9d-56a0d14c1f13 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:32.934411 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3341e4-aa51-4ec9-8b9d-56a0d14c1f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:32.934690 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3341e4-aa51-4ec9-8b9d-56a0d14c1f13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:32.935393 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3341e4-aa51-4ec9-8b9d-56a0d14c1f13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:32.937007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3341e4-aa51-4ec9-8b9d-56a0d14c1f13 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:32.950355 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 788/1583] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:32.951825 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c38197-7673-4a2d-b368-2f4c96e4e0a3 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:32.965179 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c38197-7673-4a2d-b368-2f4c96e4e0a3 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:32.972851 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe173742-b862-4a18-a26b-c3c1b41f3ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:32.974203 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe173742-b862-4a18-a26b-c3c1b41f3ee1 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:32.974837 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe173742-b862-4a18-a26b-c3c1b41f3ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:32.975692 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe173742-b862-4a18-a26b-c3c1b41f3ee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:32.980527 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe173742-b862-4a18-a26b-c3c1b41f3ee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:32.982331 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe173742-b862-4a18-a26b-c3c1b41f3ee1 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:32.996190 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 796/1584] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:33.000623 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3341e4-aa51-4ec9-8b9d-56a0d14c1f13 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:33.011953 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3341e4-aa51-4ec9-8b9d-56a0d14c1f13 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:33.017005 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73109e9a-6d43-46df-82af-5b750f65c53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:33.017691 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73109e9a-6d43-46df-82af-5b750f65c53b admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:33.017973 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73109e9a-6d43-46df-82af-5b750f65c53b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:33.018282 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73109e9a-6d43-46df-82af-5b750f65c53b admin admin] PATH_INFO: `/v3/projects/dc2bfddedf414f5bb2ab4fe0b5a32903/users/4fac5874e0674a1292715124e9b820ce/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:33.029107 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:33.043627 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73109e9a-6d43-46df-82af-5b750f65c53b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc2bfddedf414f5bb2ab4fe0b5a32903, user_id=4fac5874e0674a1292715124e9b820ce, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:33.047186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73109e9a-6d43-46df-82af-5b750f65c53b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:33.062706 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 789/1585] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:10:32 2022] PUT /identity/v3/projects/dc2bfddedf414f5bb2ab4fe0b5a32903/users/4fac5874e0674a1292715124e9b820ce/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:10:33.074567 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1d9a08-74c1-4b95-bd78-136a1ac9711e None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:33.074855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1d9a08-74c1-4b95-bd78-136a1ac9711e None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:33.075188 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1d9a08-74c1-4b95-bd78-136a1ac9711e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:33.107488 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-5f1d9a08-74c1-4b95-bd78-136a1ac9711e None None] MFA Rules not processed for user `4fac5874e0674a1292715124e9b820ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:33.107488 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-5f1d9a08-74c1-4b95-bd78-136a1ac9711e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:33.178502 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 797/1586] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:10:33.183536 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73109e9a-6d43-46df-82af-5b750f65c53b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:33.261010 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73109e9a-6d43-46df-82af-5b750f65c53b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:33.265806 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d805a97-48bd-4380-abb1-f59225a883aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:33.266411 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d805a97-48bd-4380-abb1-f59225a883aa admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:33.266691 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d805a97-48bd-4380-abb1-f59225a883aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:33.266962 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d805a97-48bd-4380-abb1-f59225a883aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:33.267633 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d805a97-48bd-4380-abb1-f59225a883aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:33.269545 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d805a97-48bd-4380-abb1-f59225a883aa admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:33.302321 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 790/1587] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:10:33 2022] POST /identity/v3/projects => generated 363 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:10:33.307105 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1d9a08-74c1-4b95-bd78-136a1ac9711e None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:33.319408 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1d9a08-74c1-4b95-bd78-136a1ac9711e None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:33.395834 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c64e3f4-3616-4dcd-8194-5bf800a212ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:33.396439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c64e3f4-3616-4dcd-8194-5bf800a212ea admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:33.396716 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c64e3f4-3616-4dcd-8194-5bf800a212ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:33.396984 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c64e3f4-3616-4dcd-8194-5bf800a212ea admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:33.397614 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c64e3f4-3616-4dcd-8194-5bf800a212ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:33.400139 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c64e3f4-3616-4dcd-8194-5bf800a212ea admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:33.422381 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 798/1588] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:10:33 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:33.426395 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d805a97-48bd-4380-abb1-f59225a883aa admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:33.437059 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d805a97-48bd-4380-abb1-f59225a883aa admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:33.442317 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a28d5f-622a-4f6c-8e12-0645ea234f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:33.442491 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a28d5f-622a-4f6c-8e12-0645ea234f78 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:33.442850 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a28d5f-622a-4f6c-8e12-0645ea234f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:33.443043 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a28d5f-622a-4f6c-8e12-0645ea234f78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:33.443704 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a28d5f-622a-4f6c-8e12-0645ea234f78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:33.445310 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a28d5f-622a-4f6c-8e12-0645ea234f78 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:33.459479 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 791/1589] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:33.460645 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c64e3f4-3616-4dcd-8194-5bf800a212ea admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:33.471494 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c64e3f4-3616-4dcd-8194-5bf800a212ea admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:33.476226 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf426a-33e0-46a1-936f-8818a843de71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:33.477475 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf426a-33e0-46a1-936f-8818a843de71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:33.477758 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf426a-33e0-46a1-936f-8818a843de71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:33.478033 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf426a-33e0-46a1-936f-8818a843de71 admin admin] PATH_INFO: `/v3/projects/60b1f4f91565479aa2d597cdc96b1cf0/users/343ff140f0d44108b15af3dce24959db/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:33.496369 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf426a-33e0-46a1-936f-8818a843de71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b1f4f91565479aa2d597cdc96b1cf0, user_id=343ff140f0d44108b15af3dce24959db, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:33.499068 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf426a-33e0-46a1-936f-8818a843de71 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:33.512507 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 799/1590] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:10:33 2022] PUT /identity/v3/projects/60b1f4f91565479aa2d597cdc96b1cf0/users/343ff140f0d44108b15af3dce24959db/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:33.517553 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a28d5f-622a-4f6c-8e12-0645ea234f78 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:33.530486 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a28d5f-622a-4f6c-8e12-0645ea234f78 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:33.594005 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a3c5d6-a8bc-4883-8c81-9b566c11454e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:33.594614 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3c5d6-a8bc-4883-8c81-9b566c11454e admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:33.594920 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3c5d6-a8bc-4883-8c81-9b566c11454e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:33.595249 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3c5d6-a8bc-4883-8c81-9b566c11454e admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:33.595910 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a3c5d6-a8bc-4883-8c81-9b566c11454e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:33.597483 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a3c5d6-a8bc-4883-8c81-9b566c11454e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:33.608010 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 792/1591] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:33 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:10:33.612147 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf426a-33e0-46a1-936f-8818a843de71 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:33.622586 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf426a-33e0-46a1-936f-8818a843de71 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:33.627138 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3cd14e-6d78-4fa3-a224-4220638272cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:33.627816 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cd14e-6d78-4fa3-a224-4220638272cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:33.628090 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cd14e-6d78-4fa3-a224-4220638272cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:33.628370 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cd14e-6d78-4fa3-a224-4220638272cb admin admin] PATH_INFO: `/v3/projects/60b1f4f91565479aa2d597cdc96b1cf0/users/343ff140f0d44108b15af3dce24959db/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:33.630483 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3cd14e-6d78-4fa3-a224-4220638272cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b1f4f91565479aa2d597cdc96b1cf0, user_id=343ff140f0d44108b15af3dce24959db, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:33.634009 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3cd14e-6d78-4fa3-a224-4220638272cb admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:33.644853 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 800/1592] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:10:33 2022] PUT /identity/v3/projects/60b1f4f91565479aa2d597cdc96b1cf0/users/343ff140f0d44108b15af3dce24959db/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:33.657052 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b3f0d-7e35-43d4-b7e0-acbd3fc883e7 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:33.657341 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b3f0d-7e35-43d4-b7e0-acbd3fc883e7 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:33.657616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b3f0d-7e35-43d4-b7e0-acbd3fc883e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:33.686780 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c60b3f0d-7e35-43d4-b7e0-acbd3fc883e7 None None] MFA Rules not processed for user `343ff140f0d44108b15af3dce24959db`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:33.688022 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c60b3f0d-7e35-43d4-b7e0-acbd3fc883e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:33.732849 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:33.759729 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 793/1593] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:33 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:33.771415 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3cd14e-6d78-4fa3-a224-4220638272cb admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:33.773107 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-9e3cd14e-6d78-4fa3-a224-4220638272cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:33.810363 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:33.838261 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3cd14e-6d78-4fa3-a224-4220638272cb admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:33.842095 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf6bfc-4f76-4564-b7f9-cbba5f73e4ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:33.842729 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf6bfc-4f76-4564-b7f9-cbba5f73e4ce service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:33.843022 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf6bfc-4f76-4564-b7f9-cbba5f73e4ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:33.843280 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf6bfc-4f76-4564-b7f9-cbba5f73e4ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:33.852930 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf6bfc-4f76-4564-b7f9-cbba5f73e4ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:33.854362 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf6bfc-4f76-4564-b7f9-cbba5f73e4ce service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:33.864095 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 801/1594] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:33 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:10:33.955452 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b3f0d-7e35-43d4-b7e0-acbd3fc883e7 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:33.964978 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b3f0d-7e35-43d4-b7e0-acbd3fc883e7 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:33.968537 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ab4d5-b5ac-4c5d-abb5-b4f6ce6d4413 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:33.969128 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ab4d5-b5ac-4c5d-abb5-b4f6ce6d4413 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:33.969405 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ab4d5-b5ac-4c5d-abb5-b4f6ce6d4413 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:33.969674 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ab4d5-b5ac-4c5d-abb5-b4f6ce6d4413 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:34.028710 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4ab4d5-b5ac-4c5d-abb5-b4f6ce6d4413 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:34.030333 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4ab4d5-b5ac-4c5d-abb5-b4f6ce6d4413 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:34.039959 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 794/1595] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:10:40.141896 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf6bfc-4f76-4564-b7f9-cbba5f73e4ce service cinder] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:40.143773 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-a4cf6bfc-4f76-4564-b7f9-cbba5f73e4ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:40.251185 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf6bfc-4f76-4564-b7f9-cbba5f73e4ce service cinder] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:40.262318 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7371250-abe9-4f91-b0a4-f8ebda721a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:40.262817 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7371250-abe9-4f91-b0a4-f8ebda721a55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:40.263119 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7371250-abe9-4f91-b0a4-f8ebda721a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:40.263396 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7371250-abe9-4f91-b0a4-f8ebda721a55 admin admin] PATH_INFO: `/v3/users/f6bfa8fad6e04c05b2c7fe8b01ab9bf1` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:40.264733 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7371250-abe9-4f91-b0a4-f8ebda721a55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6bfa8fad6e04c05b2c7fe8b01ab9bf1)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:40.266990 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7371250-abe9-4f91-b0a4-f8ebda721a55 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:40.268200 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ab4d5-b5ac-4c5d-abb5-b4f6ce6d4413 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:40.292287 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ab4d5-b5ac-4c5d-abb5-b4f6ce6d4413 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:40.299450 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca9d184-f5b2-43f1-860c-411d8c1ee947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:40.300180 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9d184-f5b2-43f1-860c-411d8c1ee947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:40.300464 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9d184-f5b2-43f1-860c-411d8c1ee947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:40.300760 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9d184-f5b2-43f1-860c-411d8c1ee947 admin admin] PATH_INFO: `/v3/users/4fac5874e0674a1292715124e9b820ce` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:40.302380 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca9d184-f5b2-43f1-860c-411d8c1ee947 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fac5874e0674a1292715124e9b820ce)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:40.304399 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca9d184-f5b2-43f1-860c-411d8c1ee947 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:40.324248 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-a7371250-abe9-4f91-b0a4-f8ebda721a55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6bfa8fad6e04c05b2c7fe8b01ab9bf1'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:40.338068 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-a7371250-abe9-4f91-b0a4-f8ebda721a55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6bfa8fad6e04c05b2c7fe8b01ab9bf1'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:40.338956 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-a7371250-abe9-4f91-b0a4-f8ebda721a55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6bfa8fad6e04c05b2c7fe8b01ab9bf1'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:40.339799 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-a7371250-abe9-4f91-b0a4-f8ebda721a55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6bfa8fad6e04c05b2c7fe8b01ab9bf1'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:40.348424 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-a7371250-abe9-4f91-b0a4-f8ebda721a55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6bfa8fad6e04c05b2c7fe8b01ab9bf1'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:40.362100 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cca9d184-f5b2-43f1-860c-411d8c1ee947 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fac5874e0674a1292715124e9b820ce'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:40.372714 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cca9d184-f5b2-43f1-860c-411d8c1ee947 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fac5874e0674a1292715124e9b820ce'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:40.373514 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cca9d184-f5b2-43f1-860c-411d8c1ee947 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fac5874e0674a1292715124e9b820ce'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:40.376137 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 802/1596] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:10:40 2022] DELETE /identity/v3/users/f6bfa8fad6e04c05b2c7fe8b01ab9bf1 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:40.380327 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18b3a8-1528-48c9-b6de-3f989c5cc0f9 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:40.380580 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18b3a8-1528-48c9-b6de-3f989c5cc0f9 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:40.380805 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18b3a8-1528-48c9-b6de-3f989c5cc0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:40.382533 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cca9d184-f5b2-43f1-860c-411d8c1ee947 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fac5874e0674a1292715124e9b820ce'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:40.398507 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cca9d184-f5b2-43f1-860c-411d8c1ee947 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fac5874e0674a1292715124e9b820ce'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:40.405274 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-4d18b3a8-1528-48c9-b6de-3f989c5cc0f9 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:40.406442 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 795/1597] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:10:40 2022] DELETE /identity/v3/users/4fac5874e0674a1292715124e9b820ce => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:40.406551 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-4d18b3a8-1528-48c9-b6de-3f989c5cc0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:40.410816 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901dedf7-9e2e-4a31-a65a-a1a6f9757c59 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:40.411018 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901dedf7-9e2e-4a31-a65a-a1a6f9757c59 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:40.411325 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901dedf7-9e2e-4a31-a65a-a1a6f9757c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:40.431489 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-901dedf7-9e2e-4a31-a65a-a1a6f9757c59 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:40.432697 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-901dedf7-9e2e-4a31-a65a-a1a6f9757c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:40.457232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:40.473189 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 803/1598] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:10:40.483331 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:40.483523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d18b3a8-1528-48c9-b6de-3f989c5cc0f9 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:40.484851 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-4d18b3a8-1528-48c9-b6de-3f989c5cc0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:40.497517 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 796/1599] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:10:40.508873 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901dedf7-9e2e-4a31-a65a-a1a6f9757c59 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:40.510306 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-901dedf7-9e2e-4a31-a65a-a1a6f9757c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:40.545141 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d18b3a8-1528-48c9-b6de-3f989c5cc0f9 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:40.548764 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415959a-1dcb-4f5d-a4db-4a856ae60c08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:40.549375 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415959a-1dcb-4f5d-a4db-4a856ae60c08 service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:40.549654 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415959a-1dcb-4f5d-a4db-4a856ae60c08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:40.549924 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415959a-1dcb-4f5d-a4db-4a856ae60c08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:40.559643 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e415959a-1dcb-4f5d-a4db-4a856ae60c08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:40.561076 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e415959a-1dcb-4f5d-a4db-4a856ae60c08 service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:40.568161 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901dedf7-9e2e-4a31-a65a-a1a6f9757c59 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:40.571352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 804/1600] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 13 13:10:40.572792 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cc525f-b915-4a43-bf81-9851f2b5dba7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:40.573386 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc525f-b915-4a43-bf81-9851f2b5dba7 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:40.573783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc525f-b915-4a43-bf81-9851f2b5dba7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:40.574063 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc525f-b915-4a43-bf81-9851f2b5dba7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:40.583923 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cc525f-b915-4a43-bf81-9851f2b5dba7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:40.585422 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cc525f-b915-4a43-bf81-9851f2b5dba7 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:40.595861 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 797/1601] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 13 13:10:41.064074 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415959a-1dcb-4f5d-a4db-4a856ae60c08 service neutron] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:41.065526 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-e415959a-1dcb-4f5d-a4db-4a856ae60c08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:41.084741 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415959a-1dcb-4f5d-a4db-4a856ae60c08 service neutron] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:41.092611 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85279229-110f-4ce6-9776-c61d5bb6868d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:41.093425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85279229-110f-4ce6-9776-c61d5bb6868d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:41.093804 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85279229-110f-4ce6-9776-c61d5bb6868d admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:41.094175 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85279229-110f-4ce6-9776-c61d5bb6868d admin admin] PATH_INFO: `/v3/projects/dc2bfddedf414f5bb2ab4fe0b5a32903` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:41.095927 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85279229-110f-4ce6-9776-c61d5bb6868d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc2bfddedf414f5bb2ab4fe0b5a32903)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:41.097774 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85279229-110f-4ce6-9776-c61d5bb6868d admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:41.174239 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-85279229-110f-4ce6-9776-c61d5bb6868d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc2bfddedf414f5bb2ab4fe0b5a32903', 'request_id': 'req-85279229-110f-4ce6-9776-c61d5bb6868d'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:41.181727 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cc525f-b915-4a43-bf81-9851f2b5dba7 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:41.183390 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a0cc525f-b915-4a43-bf81-9851f2b5dba7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:41.205903 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-85279229-110f-4ce6-9776-c61d5bb6868d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc2bfddedf414f5bb2ab4fe0b5a32903', 'request_id': 'req-85279229-110f-4ce6-9776-c61d5bb6868d'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:41.209418 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cc525f-b915-4a43-bf81-9851f2b5dba7 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:41.223958 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-85279229-110f-4ce6-9776-c61d5bb6868d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc2bfddedf414f5bb2ab4fe0b5a32903', 'request_id': 'req-85279229-110f-4ce6-9776-c61d5bb6868d'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:41.248091 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-85279229-110f-4ce6-9776-c61d5bb6868d admin admin] The token cache is being invalidate because project dc2bfddedf414f5bb2ab4fe0b5a32903 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:10:41.248612 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-85279229-110f-4ce6-9776-c61d5bb6868d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:41.249459 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-85279229-110f-4ce6-9776-c61d5bb6868d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:41.251574 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 805/1602] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:10:41 2022] DELETE /identity/v3/projects/dc2bfddedf414f5bb2ab4fe0b5a32903 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:41.256647 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85279229-110f-4ce6-9776-c61d5bb6868d admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:41.259204 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-85279229-110f-4ce6-9776-c61d5bb6868d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:41.358107 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85279229-110f-4ce6-9776-c61d5bb6868d admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:41.374021 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03caea06-8a25-4f60-87f8-7be6dcc14456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:41.376002 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03caea06-8a25-4f60-87f8-7be6dcc14456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:41.376298 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03caea06-8a25-4f60-87f8-7be6dcc14456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:41.376575 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03caea06-8a25-4f60-87f8-7be6dcc14456 admin admin] PATH_INFO: `/v3/users/343ff140f0d44108b15af3dce24959db` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:41.379979 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03caea06-8a25-4f60-87f8-7be6dcc14456 admin admin] RBAC: Authorizing `identity:delete_user(user_id=343ff140f0d44108b15af3dce24959db)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:41.387033 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924682d1-c124-4787-9f60-6b29c16fcd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:41.387739 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924682d1-c124-4787-9f60-6b29c16fcd4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:41.387992 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924682d1-c124-4787-9f60-6b29c16fcd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:41.388263 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924682d1-c124-4787-9f60-6b29c16fcd4b admin admin] PATH_INFO: `/v3/projects/a197a3a289f94205b4395a33e8dc1b0f` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:41.389781 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924682d1-c124-4787-9f60-6b29c16fcd4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a197a3a289f94205b4395a33e8dc1b0f)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:41.393499 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03caea06-8a25-4f60-87f8-7be6dcc14456 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:41.401162 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924682d1-c124-4787-9f60-6b29c16fcd4b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:41.453026 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:41.535570 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-924682d1-c124-4787-9f60-6b29c16fcd4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a197a3a289f94205b4395a33e8dc1b0f', 'request_id': 'req-924682d1-c124-4787-9f60-6b29c16fcd4b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:41.541609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-03caea06-8a25-4f60-87f8-7be6dcc14456 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '343ff140f0d44108b15af3dce24959db'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:41.560361 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-924682d1-c124-4787-9f60-6b29c16fcd4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a197a3a289f94205b4395a33e8dc1b0f', 'request_id': 'req-924682d1-c124-4787-9f60-6b29c16fcd4b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:41.569685 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-03caea06-8a25-4f60-87f8-7be6dcc14456 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '343ff140f0d44108b15af3dce24959db'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:41.571718 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-03caea06-8a25-4f60-87f8-7be6dcc14456 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '343ff140f0d44108b15af3dce24959db'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:41.572834 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-03caea06-8a25-4f60-87f8-7be6dcc14456 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '343ff140f0d44108b15af3dce24959db'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:41.577893 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-924682d1-c124-4787-9f60-6b29c16fcd4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a197a3a289f94205b4395a33e8dc1b0f', 'request_id': 'req-924682d1-c124-4787-9f60-6b29c16fcd4b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:41.591159 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-03caea06-8a25-4f60-87f8-7be6dcc14456 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '343ff140f0d44108b15af3dce24959db'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:41.602850 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-924682d1-c124-4787-9f60-6b29c16fcd4b admin admin] The token cache is being invalidate because project a197a3a289f94205b4395a33e8dc1b0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:10:41.603406 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-924682d1-c124-4787-9f60-6b29c16fcd4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:41.604066 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-924682d1-c124-4787-9f60-6b29c16fcd4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:41.605775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 798/1603] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:10:41 2022] DELETE /identity/v3/projects/a197a3a289f94205b4395a33e8dc1b0f => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:41.613646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 806/1604] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:10:41 2022] DELETE /identity/v3/users/343ff140f0d44108b15af3dce24959db => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:41.615089 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924682d1-c124-4787-9f60-6b29c16fcd4b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:41.619925 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-924682d1-c124-4787-9f60-6b29c16fcd4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:41.652079 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:41.699559 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924682d1-c124-4787-9f60-6b29c16fcd4b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:41.783664 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85717ee2-a429-4d12-92ec-2d905cd574d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:41.785516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85717ee2-a429-4d12-92ec-2d905cd574d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:41.786150 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85717ee2-a429-4d12-92ec-2d905cd574d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:41.791820 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85717ee2-a429-4d12-92ec-2d905cd574d3 admin admin] PATH_INFO: `/v3/users/283f83f1bb2f4eb0b3f77418fd6d0333` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:41.793166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85717ee2-a429-4d12-92ec-2d905cd574d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=283f83f1bb2f4eb0b3f77418fd6d0333)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:41.795178 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85717ee2-a429-4d12-92ec-2d905cd574d3 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:41.861519 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-85717ee2-a429-4d12-92ec-2d905cd574d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '283f83f1bb2f4eb0b3f77418fd6d0333'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:41.872811 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-85717ee2-a429-4d12-92ec-2d905cd574d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '283f83f1bb2f4eb0b3f77418fd6d0333'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:41.873716 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-85717ee2-a429-4d12-92ec-2d905cd574d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '283f83f1bb2f4eb0b3f77418fd6d0333'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:41.885397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-85717ee2-a429-4d12-92ec-2d905cd574d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '283f83f1bb2f4eb0b3f77418fd6d0333'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:41.902709 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-85717ee2-a429-4d12-92ec-2d905cd574d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '283f83f1bb2f4eb0b3f77418fd6d0333'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:41.909490 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 799/1605] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:10:41 2022] DELETE /identity/v3/users/283f83f1bb2f4eb0b3f77418fd6d0333 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:41.970639 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03caea06-8a25-4f60-87f8-7be6dcc14456 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:41.973115 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-03caea06-8a25-4f60-87f8-7be6dcc14456 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:41.990721 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03caea06-8a25-4f60-87f8-7be6dcc14456 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:42.100955 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c12bb5-2eab-4425-a461-2fa8956d762a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:42.104273 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c12bb5-2eab-4425-a461-2fa8956d762a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:42.104794 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c12bb5-2eab-4425-a461-2fa8956d762a admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:42.105209 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c12bb5-2eab-4425-a461-2fa8956d762a admin admin] PATH_INFO: `/v3/projects/60b1f4f91565479aa2d597cdc96b1cf0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:42.106550 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c12bb5-2eab-4425-a461-2fa8956d762a admin admin] RBAC: Authorizing `identity:delete_project(project_id=60b1f4f91565479aa2d597cdc96b1cf0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:42.108966 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c12bb5-2eab-4425-a461-2fa8956d762a admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:42.195150 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c2c12bb5-2eab-4425-a461-2fa8956d762a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60b1f4f91565479aa2d597cdc96b1cf0', 'request_id': 'req-c2c12bb5-2eab-4425-a461-2fa8956d762a'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:42.202237 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c2c12bb5-2eab-4425-a461-2fa8956d762a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60b1f4f91565479aa2d597cdc96b1cf0', 'request_id': 'req-c2c12bb5-2eab-4425-a461-2fa8956d762a'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:42.211723 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c2c12bb5-2eab-4425-a461-2fa8956d762a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60b1f4f91565479aa2d597cdc96b1cf0', 'request_id': 'req-c2c12bb5-2eab-4425-a461-2fa8956d762a'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:42.231096 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c2c12bb5-2eab-4425-a461-2fa8956d762a admin admin] The token cache is being invalidate because project 60b1f4f91565479aa2d597cdc96b1cf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:10:42.231396 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c2c12bb5-2eab-4425-a461-2fa8956d762a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:42.232562 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c2c12bb5-2eab-4425-a461-2fa8956d762a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:42.234355 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 807/1606] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:10:41 2022] DELETE /identity/v3/projects/60b1f4f91565479aa2d597cdc96b1cf0 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:42.239809 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ac09a-170b-4715-a46d-e05b1f03e371 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:42.241153 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ac09a-170b-4715-a46d-e05b1f03e371 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:42.243813 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ac09a-170b-4715-a46d-e05b1f03e371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:42.268359 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6e9ac09a-170b-4715-a46d-e05b1f03e371 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:42.269621 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6e9ac09a-170b-4715-a46d-e05b1f03e371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:42.323755 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:42.345868 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 800/1607] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:42.347918 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d0120-0d0f-4b8e-9d33-889e5a160e50 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:42.348281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d0120-0d0f-4b8e-9d33-889e5a160e50 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:42.348648 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d0120-0d0f-4b8e-9d33-889e5a160e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:42.370682 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-5d5d0120-0d0f-4b8e-9d33-889e5a160e50 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:42.372073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-5d5d0120-0d0f-4b8e-9d33-889e5a160e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:42.383517 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 808/1608] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:42.393128 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ac09a-170b-4715-a46d-e05b1f03e371 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:42.407426 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ac09a-170b-4715-a46d-e05b1f03e371 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:42.416172 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c1d28a-7a2e-48a5-9d21-52779cfd7874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:42.417103 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1d28a-7a2e-48a5-9d21-52779cfd7874 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:42.417568 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1d28a-7a2e-48a5-9d21-52779cfd7874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:42.418083 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1d28a-7a2e-48a5-9d21-52779cfd7874 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:42.418958 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c1d28a-7a2e-48a5-9d21-52779cfd7874 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:10:42.419427 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c1d28a-7a2e-48a5-9d21-52779cfd7874 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:42.421175 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c1d28a-7a2e-48a5-9d21-52779cfd7874 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:42.444484 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 801/1609] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:10:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:10:42.447408 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d0120-0d0f-4b8e-9d33-889e5a160e50 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:42.449291 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-5d5d0120-0d0f-4b8e-9d33-889e5a160e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:42.450050 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c1d28a-7a2e-48a5-9d21-52779cfd7874 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:42.470599 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c1d28a-7a2e-48a5-9d21-52779cfd7874 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:42.474741 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d0120-0d0f-4b8e-9d33-889e5a160e50 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:42.484066 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec19549-4568-4a7f-91e5-82421781d2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:42.484717 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec19549-4568-4a7f-91e5-82421781d2c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:42.484993 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec19549-4568-4a7f-91e5-82421781d2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:42.485268 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec19549-4568-4a7f-91e5-82421781d2c0 admin admin] PATH_INFO: `/v3/projects/ffa7f718542d4898abc2b8d5fcbe7bf1` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:42.486404 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec19549-4568-4a7f-91e5-82421781d2c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffa7f718542d4898abc2b8d5fcbe7bf1)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:42.488510 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec19549-4568-4a7f-91e5-82421781d2c0 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:42.498038 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e613be5-28a1-4d05-8b16-e33bff900a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:42.498867 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e613be5-28a1-4d05-8b16-e33bff900a6b admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:42.499179 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e613be5-28a1-4d05-8b16-e33bff900a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:42.499456 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e613be5-28a1-4d05-8b16-e33bff900a6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:42.500465 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e613be5-28a1-4d05-8b16-e33bff900a6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:42.503641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e613be5-28a1-4d05-8b16-e33bff900a6b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:42.526618 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:42.588729 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-4ec19549-4568-4a7f-91e5-82421781d2c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffa7f718542d4898abc2b8d5fcbe7bf1', 'request_id': 'req-4ec19549-4568-4a7f-91e5-82421781d2c0'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:42.600037 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 802/1610] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:10:42 2022] POST /identity/v3/projects => generated 383 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:42.604712 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e613be5-28a1-4d05-8b16-e33bff900a6b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:42.606359 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-4ec19549-4568-4a7f-91e5-82421781d2c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffa7f718542d4898abc2b8d5fcbe7bf1', 'request_id': 'req-4ec19549-4568-4a7f-91e5-82421781d2c0'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:42.636692 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-4ec19549-4568-4a7f-91e5-82421781d2c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffa7f718542d4898abc2b8d5fcbe7bf1', 'request_id': 'req-4ec19549-4568-4a7f-91e5-82421781d2c0'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:42.664698 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-4ec19549-4568-4a7f-91e5-82421781d2c0 admin admin] The token cache is being invalidate because project ffa7f718542d4898abc2b8d5fcbe7bf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:10:42.664698 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-4ec19549-4568-4a7f-91e5-82421781d2c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:42.665339 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-4ec19549-4568-4a7f-91e5-82421781d2c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:42.668131 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 809/1611] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:10:42 2022] DELETE /identity/v3/projects/ffa7f718542d4898abc2b8d5fcbe7bf1 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:42.674983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2df14-6754-43de-aa97-57c7ccc915ed None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:42.674983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2df14-6754-43de-aa97-57c7ccc915ed None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:42.676244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2df14-6754-43de-aa97-57c7ccc915ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:42.734567 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-2de2df14-6754-43de-aa97-57c7ccc915ed None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:42.734567 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-2de2df14-6754-43de-aa97-57c7ccc915ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:42.767101 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e613be5-28a1-4d05-8b16-e33bff900a6b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:42.794872 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164af2a-f053-4fd0-b6c1-0d74959d5842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:42.795548 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164af2a-f053-4fd0-b6c1-0d74959d5842 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:42.795831 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164af2a-f053-4fd0-b6c1-0d74959d5842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:42.796109 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164af2a-f053-4fd0-b6c1-0d74959d5842 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:42.796760 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e164af2a-f053-4fd0-b6c1-0d74959d5842 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:42.798830 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e164af2a-f053-4fd0-b6c1-0d74959d5842 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:42.837695 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 803/1612] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:10:42 2022] POST /identity/v3/users => generated 292 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:10:42.848090 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164af2a-f053-4fd0-b6c1-0d74959d5842 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:42.859006 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e164af2a-f053-4fd0-b6c1-0d74959d5842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:42.863458 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 810/1613] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:42.873722 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335bc123-4256-47c1-bc81-53a0f6410a43 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:42.873857 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335bc123-4256-47c1-bc81-53a0f6410a43 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:42.874166 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335bc123-4256-47c1-bc81-53a0f6410a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:42.877565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164af2a-f053-4fd0-b6c1-0d74959d5842 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:42.902998 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820bc206-16cd-4aa4-82b1-658a0dfbbab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:42.904072 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820bc206-16cd-4aa4-82b1-658a0dfbbab2 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:42.904519 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820bc206-16cd-4aa4-82b1-658a0dfbbab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:42.904964 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820bc206-16cd-4aa4-82b1-658a0dfbbab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:42.905778 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820bc206-16cd-4aa4-82b1-658a0dfbbab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:42.907711 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820bc206-16cd-4aa4-82b1-658a0dfbbab2 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:42.919861 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 804/1614] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:42 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:42.923628 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820bc206-16cd-4aa4-82b1-658a0dfbbab2 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:42.931298 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-335bc123-4256-47c1-bc81-53a0f6410a43 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:42.932525 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-335bc123-4256-47c1-bc81-53a0f6410a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:42.944333 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 811/1615] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:42.944506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820bc206-16cd-4aa4-82b1-658a0dfbbab2 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:42.945993 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335bc123-4256-47c1-bc81-53a0f6410a43 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:42.948679 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a378fe-e340-4ffc-848c-e3db9764d35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:42.950227 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a378fe-e340-4ffc-848c-e3db9764d35d admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:42.950682 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a378fe-e340-4ffc-848c-e3db9764d35d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:42.951166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a378fe-e340-4ffc-848c-e3db9764d35d admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:42.952166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a378fe-e340-4ffc-848c-e3db9764d35d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:42.953654 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a378fe-e340-4ffc-848c-e3db9764d35d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:42.961504 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335bc123-4256-47c1-bc81-53a0f6410a43 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:42.966260 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 805/1616] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:42 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:42.970314 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a378fe-e340-4ffc-848c-e3db9764d35d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:42.980117 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a4046-a291-4d4d-8cad-76cb7cd78099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:42.981546 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a4046-a291-4d4d-8cad-76cb7cd78099 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:42.982151 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a4046-a291-4d4d-8cad-76cb7cd78099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:42.982592 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a4046-a291-4d4d-8cad-76cb7cd78099 admin admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:42.983437 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1a4046-a291-4d4d-8cad-76cb7cd78099 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:10:42.983887 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1a4046-a291-4d4d-8cad-76cb7cd78099 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:42.985322 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a378fe-e340-4ffc-848c-e3db9764d35d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:42.985867 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1a4046-a291-4d4d-8cad-76cb7cd78099 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:42.991178 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49e199-2a30-4aa0-8636-9f0bafaf28e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:42.991942 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49e199-2a30-4aa0-8636-9f0bafaf28e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:42.992380 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49e199-2a30-4aa0-8636-9f0bafaf28e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:42.992695 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49e199-2a30-4aa0-8636-9f0bafaf28e8 admin admin] PATH_INFO: `/v3/projects/9a98a16c5206459ea156af4a4e52b0f3/users/e30245dde46648b9b22d67913281f769/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:42.999766 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 812/1617] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:10:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:10:43.004663 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a4046-a291-4d4d-8cad-76cb7cd78099 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:43.013576 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49e199-2a30-4aa0-8636-9f0bafaf28e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a98a16c5206459ea156af4a4e52b0f3, user_id=e30245dde46648b9b22d67913281f769, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:43.017163 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49e199-2a30-4aa0-8636-9f0bafaf28e8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:43.018443 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a4046-a291-4d4d-8cad-76cb7cd78099 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:43.026009 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eed0541-639b-48ac-beae-8b56a39e7809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:43.027377 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed0541-639b-48ac-beae-8b56a39e7809 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:43.027663 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed0541-639b-48ac-beae-8b56a39e7809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:43.028061 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed0541-639b-48ac-beae-8b56a39e7809 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:43.028819 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eed0541-639b-48ac-beae-8b56a39e7809 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:43.031028 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eed0541-639b-48ac-beae-8b56a39e7809 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:43.035471 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 806/1618] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:10:42 2022] PUT /identity/v3/projects/9a98a16c5206459ea156af4a4e52b0f3/users/e30245dde46648b9b22d67913281f769/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:10:43.048353 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c168d0-3782-450a-a500-c7e86ff1ea6d None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:43.049089 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c168d0-3782-450a-a500-c7e86ff1ea6d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:43.049296 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c168d0-3782-450a-a500-c7e86ff1ea6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:43.073051 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 813/1619] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:10:43 2022] POST /identity/v3/projects => generated 361 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:43.074083 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eed0541-639b-48ac-beae-8b56a39e7809 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:43.086567 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-57c168d0-3782-450a-a500-c7e86ff1ea6d None None] MFA Rules not processed for user `e30245dde46648b9b22d67913281f769`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:43.089778 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-57c168d0-3782-450a-a500-c7e86ff1ea6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:43.140447 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eed0541-639b-48ac-beae-8b56a39e7809 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:43.142105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:43.145410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de89fa-968b-41fd-bc99-61d292892cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:43.146058 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de89fa-968b-41fd-bc99-61d292892cc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:43.146304 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de89fa-968b-41fd-bc99-61d292892cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:43.146585 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de89fa-968b-41fd-bc99-61d292892cc9 admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:43.147302 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5de89fa-968b-41fd-bc99-61d292892cc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:43.149832 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5de89fa-968b-41fd-bc99-61d292892cc9 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:43.160180 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:43.168761 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 807/1620] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:10:43.178503 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 814/1621] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:10:43 2022] POST /identity/v3/users => generated 281 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:10:43.180624 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c168d0-3782-450a-a500-c7e86ff1ea6d None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:43.182299 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de89fa-968b-41fd-bc99-61d292892cc9 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:43.183912 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-f5de89fa-968b-41fd-bc99-61d292892cc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:43.191781 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c168d0-3782-450a-a500-c7e86ff1ea6d None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:43.196971 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e98336-97f4-49fa-a95e-6b230d68b087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:43.197697 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e98336-97f4-49fa-a95e-6b230d68b087 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:43.197978 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e98336-97f4-49fa-a95e-6b230d68b087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:43.198309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e98336-97f4-49fa-a95e-6b230d68b087 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:43.199083 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e98336-97f4-49fa-a95e-6b230d68b087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:43.203092 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e98336-97f4-49fa-a95e-6b230d68b087 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:43.213188 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 808/1622] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:43.216989 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e98336-97f4-49fa-a95e-6b230d68b087 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:43.228453 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e98336-97f4-49fa-a95e-6b230d68b087 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:43.233129 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efd33fa-74ca-4b3f-a31a-b5f3d97e35d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:43.234107 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd33fa-74ca-4b3f-a31a-b5f3d97e35d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:43.234392 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd33fa-74ca-4b3f-a31a-b5f3d97e35d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:43.234665 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd33fa-74ca-4b3f-a31a-b5f3d97e35d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:43.235341 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efd33fa-74ca-4b3f-a31a-b5f3d97e35d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:43.236942 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efd33fa-74ca-4b3f-a31a-b5f3d97e35d9 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:43.244163 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de89fa-968b-41fd-bc99-61d292892cc9 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:43.246921 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 809/1623] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:10:43.247683 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d700c-ad3f-44fa-bc01-fde57f029c61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:43.248281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d700c-ad3f-44fa-bc01-fde57f029c61 service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:43.248556 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d700c-ad3f-44fa-bc01-fde57f029c61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:43.248828 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d700c-ad3f-44fa-bc01-fde57f029c61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:43.250954 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efd33fa-74ca-4b3f-a31a-b5f3d97e35d9 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:43.257500 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2d700c-ad3f-44fa-bc01-fde57f029c61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:43.259028 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2d700c-ad3f-44fa-bc01-fde57f029c61 service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:43.264431 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efd33fa-74ca-4b3f-a31a-b5f3d97e35d9 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:43.268343 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 815/1624] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:10:43.269129 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10e5cc-1d9b-44a4-b507-3d7de9395688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:43.269822 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10e5cc-1d9b-44a4-b507-3d7de9395688 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:43.270062 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10e5cc-1d9b-44a4-b507-3d7de9395688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:43.270334 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10e5cc-1d9b-44a4-b507-3d7de9395688 admin admin] PATH_INFO: `/v3/projects/224d83acc9644bdcbcf0ed0800538516/users/b96fe23e2961414f83e0d90bb2fc3c17/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:43.285925 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef10e5cc-1d9b-44a4-b507-3d7de9395688 admin admin] RBAC: Authorizing `identity:create_grant(project_id=224d83acc9644bdcbcf0ed0800538516, user_id=b96fe23e2961414f83e0d90bb2fc3c17, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:43.288708 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef10e5cc-1d9b-44a4-b507-3d7de9395688 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:43.291506 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d700c-ad3f-44fa-bc01-fde57f029c61 service cinder] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:43.302338 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d700c-ad3f-44fa-bc01-fde57f029c61 service cinder] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:43.302810 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 810/1625] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:10:43 2022] PUT /identity/v3/projects/224d83acc9644bdcbcf0ed0800538516/users/b96fe23e2961414f83e0d90bb2fc3c17/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:10:43.313051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ac0fb-961d-4e22-b7af-65429cc8633c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:43.313334 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ac0fb-961d-4e22-b7af-65429cc8633c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:43.313606 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ac0fb-961d-4e22-b7af-65429cc8633c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:43.347289 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8d9ac0fb-961d-4e22-b7af-65429cc8633c None None] MFA Rules not processed for user `b96fe23e2961414f83e0d90bb2fc3c17`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:43.348582 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8d9ac0fb-961d-4e22-b7af-65429cc8633c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:43.363504 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee7680-7922-4c49-9f6d-f47fdb47b9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:43.364135 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee7680-7922-4c49-9f6d-f47fdb47b9ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:43.364415 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee7680-7922-4c49-9f6d-f47fdb47b9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:43.364687 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee7680-7922-4c49-9f6d-f47fdb47b9ee admin admin] PATH_INFO: `/v3/users/e30245dde46648b9b22d67913281f769` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:43.365825 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ee7680-7922-4c49-9f6d-f47fdb47b9ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=e30245dde46648b9b22d67913281f769)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:43.367823 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ee7680-7922-4c49-9f6d-f47fdb47b9ee admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:43.414748 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-78ee7680-7922-4c49-9f6d-f47fdb47b9ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e30245dde46648b9b22d67913281f769'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:43.418878 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 811/1626] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:10:43.422104 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-78ee7680-7922-4c49-9f6d-f47fdb47b9ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e30245dde46648b9b22d67913281f769'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:43.423211 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-78ee7680-7922-4c49-9f6d-f47fdb47b9ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e30245dde46648b9b22d67913281f769'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:43.423859 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-78ee7680-7922-4c49-9f6d-f47fdb47b9ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e30245dde46648b9b22d67913281f769'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:43.431921 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-78ee7680-7922-4c49-9f6d-f47fdb47b9ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e30245dde46648b9b22d67913281f769'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:43.450987 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 816/1627] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:10:43 2022] DELETE /identity/v3/users/e30245dde46648b9b22d67913281f769 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:10:43.455891 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b839f-cf11-49ca-bf9d-da144e106f43 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:43.456180 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b839f-cf11-49ca-bf9d-da144e106f43 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:43.456514 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b839f-cf11-49ca-bf9d-da144e106f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:43.461851 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee7680-7922-4c49-9f6d-f47fdb47b9ee admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:43.463278 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-78ee7680-7922-4c49-9f6d-f47fdb47b9ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:43.477432 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d65b839f-cf11-49ca-bf9d-da144e106f43 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:43.478660 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d65b839f-cf11-49ca-bf9d-da144e106f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:43.485629 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:43.537752 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee7680-7922-4c49-9f6d-f47fdb47b9ee admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:43.543228 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e23185-a159-4bd2-b081-23084791706b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:43.544302 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e23185-a159-4bd2-b081-23084791706b service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:43.544808 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e23185-a159-4bd2-b081-23084791706b service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:43.545498 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e23185-a159-4bd2-b081-23084791706b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:43.553263 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-71e23185-a159-4bd2-b081-23084791706b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:43.574761 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 812/1628] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:43.585378 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b839f-cf11-49ca-bf9d-da144e106f43 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:43.586841 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d65b839f-cf11-49ca-bf9d-da144e106f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:43.640472 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e23185-a159-4bd2-b081-23084791706b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:43.641935 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e23185-a159-4bd2-b081-23084791706b service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:43.652608 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:43.654460 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 817/1629] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 13 13:10:43.671962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e23185-a159-4bd2-b081-23084791706b service cinder] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:43.674413 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-71e23185-a159-4bd2-b081-23084791706b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:43.692175 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b839f-cf11-49ca-bf9d-da144e106f43 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:43.699001 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e23185-a159-4bd2-b081-23084791706b service cinder] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:43.701576 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc3af3-98c6-4821-9732-b3c1e43ed78a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:43.702204 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc3af3-98c6-4821-9732-b3c1e43ed78a service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:43.702479 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc3af3-98c6-4821-9732-b3c1e43ed78a service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:43.702874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc3af3-98c6-4821-9732-b3c1e43ed78a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:43.711912 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89611944-8d1a-4cc8-b307-61c7f1c7adca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:43.713046 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89611944-8d1a-4cc8-b307-61c7f1c7adca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:43.713243 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89611944-8d1a-4cc8-b307-61c7f1c7adca admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:43.713619 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89611944-8d1a-4cc8-b307-61c7f1c7adca admin admin] PATH_INFO: `/v3/users/b96fe23e2961414f83e0d90bb2fc3c17` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:43.714932 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89611944-8d1a-4cc8-b307-61c7f1c7adca admin admin] RBAC: Authorizing `identity:delete_user(user_id=b96fe23e2961414f83e0d90bb2fc3c17)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:43.717360 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89611944-8d1a-4cc8-b307-61c7f1c7adca admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:43.734698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc3af3-98c6-4821-9732-b3c1e43ed78a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:43.736170 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc3af3-98c6-4821-9732-b3c1e43ed78a service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:43.749088 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 813/1630] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 13 13:10:43.799280 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-89611944-8d1a-4cc8-b307-61c7f1c7adca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b96fe23e2961414f83e0d90bb2fc3c17'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:43.814831 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-89611944-8d1a-4cc8-b307-61c7f1c7adca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b96fe23e2961414f83e0d90bb2fc3c17'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:43.816201 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-89611944-8d1a-4cc8-b307-61c7f1c7adca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b96fe23e2961414f83e0d90bb2fc3c17'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:43.817135 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-89611944-8d1a-4cc8-b307-61c7f1c7adca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b96fe23e2961414f83e0d90bb2fc3c17'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:43.832529 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-89611944-8d1a-4cc8-b307-61c7f1c7adca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b96fe23e2961414f83e0d90bb2fc3c17'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:43.861306 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 818/1631] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:10:43 2022] DELETE /identity/v3/users/b96fe23e2961414f83e0d90bb2fc3c17 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:43.866728 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addb943-2f29-412b-871c-96a5be2da0f7 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:43.867193 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addb943-2f29-412b-871c-96a5be2da0f7 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:43.867617 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addb943-2f29-412b-871c-96a5be2da0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:43.890676 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6addb943-2f29-412b-871c-96a5be2da0f7 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:43.892009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6addb943-2f29-412b-871c-96a5be2da0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:44.047004 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 814/1632] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:44.058578 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89611944-8d1a-4cc8-b307-61c7f1c7adca admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:44.063039 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-89611944-8d1a-4cc8-b307-61c7f1c7adca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:44.122476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:44.140928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89611944-8d1a-4cc8-b307-61c7f1c7adca admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:44.148095 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e2b4c-740d-419c-b36b-ea359d8189fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:44.149985 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e2b4c-740d-419c-b36b-ea359d8189fd service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:44.150448 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e2b4c-740d-419c-b36b-ea359d8189fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:44.151014 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e2b4c-740d-419c-b36b-ea359d8189fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:44.164290 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16e2b4c-740d-419c-b36b-ea359d8189fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:44.166032 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16e2b4c-740d-419c-b36b-ea359d8189fd service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:44.182627 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 819/1633] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:10:44.361727 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6addb943-2f29-412b-871c-96a5be2da0f7 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:44.363484 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6addb943-2f29-412b-871c-96a5be2da0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:44.378591 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6addb943-2f29-412b-871c-96a5be2da0f7 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:44.384979 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f1f79-ce6e-4511-a6dc-126a265b00f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:44.385743 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f1f79-ce6e-4511-a6dc-126a265b00f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:44.386183 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f1f79-ce6e-4511-a6dc-126a265b00f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:44.386547 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f1f79-ce6e-4511-a6dc-126a265b00f9 admin admin] PATH_INFO: `/v3/projects/9a98a16c5206459ea156af4a4e52b0f3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:44.388199 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64f1f79-ce6e-4511-a6dc-126a265b00f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a98a16c5206459ea156af4a4e52b0f3)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:44.391075 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64f1f79-ce6e-4511-a6dc-126a265b00f9 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:44.493500 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d64f1f79-ce6e-4511-a6dc-126a265b00f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a98a16c5206459ea156af4a4e52b0f3', 'request_id': 'req-d64f1f79-ce6e-4511-a6dc-126a265b00f9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:44.510028 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d64f1f79-ce6e-4511-a6dc-126a265b00f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a98a16c5206459ea156af4a4e52b0f3', 'request_id': 'req-d64f1f79-ce6e-4511-a6dc-126a265b00f9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:44.519861 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d64f1f79-ce6e-4511-a6dc-126a265b00f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a98a16c5206459ea156af4a4e52b0f3', 'request_id': 'req-d64f1f79-ce6e-4511-a6dc-126a265b00f9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:44.633414 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d64f1f79-ce6e-4511-a6dc-126a265b00f9 admin admin] The token cache is being invalidate because project 9a98a16c5206459ea156af4a4e52b0f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:10:44.639422 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d64f1f79-ce6e-4511-a6dc-126a265b00f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:44.647093 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d64f1f79-ce6e-4511-a6dc-126a265b00f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:44.647093 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 815/1634] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:10:44 2022] DELETE /identity/v3/projects/9a98a16c5206459ea156af4a4e52b0f3 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:44.651999 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38c6c5-e503-46be-9674-ce7f4f103b72 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:44.651999 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38c6c5-e503-46be-9674-ce7f4f103b72 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:44.651999 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38c6c5-e503-46be-9674-ce7f4f103b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:44.697691 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-bd38c6c5-e503-46be-9674-ce7f4f103b72 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:44.699047 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-bd38c6c5-e503-46be-9674-ce7f4f103b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:44.832890 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 820/1635] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:10:44.833290 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f1f79-ce6e-4511-a6dc-126a265b00f9 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:44.835715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d64f1f79-ce6e-4511-a6dc-126a265b00f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:44.841853 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a0c4f-462b-4e50-8aa8-0e7785216996 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:44.842162 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a0c4f-462b-4e50-8aa8-0e7785216996 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:44.842445 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a0c4f-462b-4e50-8aa8-0e7785216996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:44.858699 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f1f79-ce6e-4511-a6dc-126a265b00f9 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:44.865128 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-c95a0c4f-462b-4e50-8aa8-0e7785216996 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:44.866409 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-c95a0c4f-462b-4e50-8aa8-0e7785216996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:44.872447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e294448-8501-4fdd-8bdc-237cfb68c9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:44.873518 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e294448-8501-4fdd-8bdc-237cfb68c9e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:44.874033 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e294448-8501-4fdd-8bdc-237cfb68c9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:44.874449 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e294448-8501-4fdd-8bdc-237cfb68c9e8 admin admin] PATH_INFO: `/v3/projects/224d83acc9644bdcbcf0ed0800538516` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:44.876886 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 821/1636] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:10:44.878841 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e294448-8501-4fdd-8bdc-237cfb68c9e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=224d83acc9644bdcbcf0ed0800538516)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:44.881082 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a0c4f-462b-4e50-8aa8-0e7785216996 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:44.883568 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e294448-8501-4fdd-8bdc-237cfb68c9e8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:44.891787 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a0c4f-462b-4e50-8aa8-0e7785216996 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:44.896590 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5773ac99-e67e-4118-849d-ffaaaa73345e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:44.897203 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773ac99-e67e-4118-849d-ffaaaa73345e admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:44.897482 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773ac99-e67e-4118-849d-ffaaaa73345e admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:44.897754 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773ac99-e67e-4118-849d-ffaaaa73345e admin admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:44.898449 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5773ac99-e67e-4118-849d-ffaaaa73345e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:10:44.898761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5773ac99-e67e-4118-849d-ffaaaa73345e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:44.900598 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5773ac99-e67e-4118-849d-ffaaaa73345e admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:44.913070 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 822/1637] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:10:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:10:44.926025 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5773ac99-e67e-4118-849d-ffaaaa73345e admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:44.943260 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5773ac99-e67e-4118-849d-ffaaaa73345e admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:44.961651 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4e294448-8501-4fdd-8bdc-237cfb68c9e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '224d83acc9644bdcbcf0ed0800538516', 'request_id': 'req-4e294448-8501-4fdd-8bdc-237cfb68c9e8'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:44.969089 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:44.973279 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4e294448-8501-4fdd-8bdc-237cfb68c9e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '224d83acc9644bdcbcf0ed0800538516', 'request_id': 'req-4e294448-8501-4fdd-8bdc-237cfb68c9e8'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:44.990351 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4e294448-8501-4fdd-8bdc-237cfb68c9e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '224d83acc9644bdcbcf0ed0800538516', 'request_id': 'req-4e294448-8501-4fdd-8bdc-237cfb68c9e8'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:45.009401 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4e294448-8501-4fdd-8bdc-237cfb68c9e8 admin admin] The token cache is being invalidate because project 224d83acc9644bdcbcf0ed0800538516 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:10:45.009696 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4e294448-8501-4fdd-8bdc-237cfb68c9e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:45.010406 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4e294448-8501-4fdd-8bdc-237cfb68c9e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:45.015203 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 816/1638] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:10:44 2022] DELETE /identity/v3/projects/224d83acc9644bdcbcf0ed0800538516 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:45.017720 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fed71c-e3bb-4cc4-895f-21ace91d6854 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:45.018621 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fed71c-e3bb-4cc4-895f-21ace91d6854 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:45.018621 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fed71c-e3bb-4cc4-895f-21ace91d6854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:45.040831 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d3fed71c-e3bb-4cc4-895f-21ace91d6854 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:45.041883 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6dbc0a-f76b-454c-a383-55e56c1e2a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:45.042033 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d3fed71c-e3bb-4cc4-895f-21ace91d6854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:45.042648 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6dbc0a-f76b-454c-a383-55e56c1e2a3d admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:45.043511 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6dbc0a-f76b-454c-a383-55e56c1e2a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:45.043994 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6dbc0a-f76b-454c-a383-55e56c1e2a3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:45.044647 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6dbc0a-f76b-454c-a383-55e56c1e2a3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:45.046709 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6dbc0a-f76b-454c-a383-55e56c1e2a3d admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:45.050822 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 817/1639] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:45.060350 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb01c4-0503-4bb4-a546-90f93edf8126 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:45.060639 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb01c4-0503-4bb4-a546-90f93edf8126 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:45.060930 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb01c4-0503-4bb4-a546-90f93edf8126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:45.080197 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 823/1640] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:10:44 2022] POST /identity/v3/projects => generated 365 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:45.084009 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6dbc0a-f76b-454c-a383-55e56c1e2a3d admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:45.084647 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-65cb01c4-0503-4bb4-a546-90f93edf8126 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:45.085923 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-65cb01c4-0503-4bb4-a546-90f93edf8126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:45.086971 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-9b6dbc0a-f76b-454c-a383-55e56c1e2a3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:45.149170 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:45.163405 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 818/1641] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:45.165116 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb01c4-0503-4bb4-a546-90f93edf8126 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:45.167288 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6dbc0a-f76b-454c-a383-55e56c1e2a3d admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:45.172977 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873658c5-a19f-4fb0-99f1-273eea9d9665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:45.173703 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873658c5-a19f-4fb0-99f1-273eea9d9665 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:45.173992 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873658c5-a19f-4fb0-99f1-273eea9d9665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:45.174270 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873658c5-a19f-4fb0-99f1-273eea9d9665 admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:45.174917 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873658c5-a19f-4fb0-99f1-273eea9d9665 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:45.175488 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb01c4-0503-4bb4-a546-90f93edf8126 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:45.177199 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873658c5-a19f-4fb0-99f1-273eea9d9665 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:45.180505 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d5e2f-b3e2-4882-9c91-900a43ce8a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:45.181248 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d5e2f-b3e2-4882-9c91-900a43ce8a03 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:45.181472 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d5e2f-b3e2-4882-9c91-900a43ce8a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:45.181743 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d5e2f-b3e2-4882-9c91-900a43ce8a03 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:45.182476 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1d5e2f-b3e2-4882-9c91-900a43ce8a03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:10:45.182784 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1d5e2f-b3e2-4882-9c91-900a43ce8a03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:45.184430 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1d5e2f-b3e2-4882-9c91-900a43ce8a03 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:45.202705 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 819/1642] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:10:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:10:45.204531 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d5e2f-b3e2-4882-9c91-900a43ce8a03 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:45.212837 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 824/1643] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:10:45 2022] POST /identity/v3/users => generated 283 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:10:45.214916 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873658c5-a19f-4fb0-99f1-273eea9d9665 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:45.219723 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d5e2f-b3e2-4882-9c91-900a43ce8a03 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:45.225751 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873658c5-a19f-4fb0-99f1-273eea9d9665 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:45.227301 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb9d78e-cbb0-4ba2-9b87-bddce7216169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:45.227928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9d78e-cbb0-4ba2-9b87-bddce7216169 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:45.228338 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9d78e-cbb0-4ba2-9b87-bddce7216169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:45.229024 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9d78e-cbb0-4ba2-9b87-bddce7216169 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:45.229940 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb9d78e-cbb0-4ba2-9b87-bddce7216169 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:45.231453 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d892247-a153-4f79-a6d8-5ed56a1f9e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:45.232066 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb9d78e-cbb0-4ba2-9b87-bddce7216169 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:45.232185 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d892247-a153-4f79-a6d8-5ed56a1f9e24 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:45.232365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d892247-a153-4f79-a6d8-5ed56a1f9e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:45.232611 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d892247-a153-4f79-a6d8-5ed56a1f9e24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:45.233259 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d892247-a153-4f79-a6d8-5ed56a1f9e24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:45.236160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d892247-a153-4f79-a6d8-5ed56a1f9e24 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:45.247954 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 825/1644] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:10:45.252939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d892247-a153-4f79-a6d8-5ed56a1f9e24 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:45.263665 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d892247-a153-4f79-a6d8-5ed56a1f9e24 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:45.268567 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd68879-2051-41f9-9721-9d62a9c17c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:45.272377 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd68879-2051-41f9-9721-9d62a9c17c3d admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:45.272535 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 820/1645] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:10:45 2022] POST /identity/v3/projects => generated 369 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:10:45.272630 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd68879-2051-41f9-9721-9d62a9c17c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:45.272630 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd68879-2051-41f9-9721-9d62a9c17c3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:45.272630 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd68879-2051-41f9-9721-9d62a9c17c3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:45.272813 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd68879-2051-41f9-9721-9d62a9c17c3d admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:45.274365 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb9d78e-cbb0-4ba2-9b87-bddce7216169 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:45.285025 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 826/1646] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:10:45.288975 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd68879-2051-41f9-9721-9d62a9c17c3d admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:45.300304 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd68879-2051-41f9-9721-9d62a9c17c3d admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:45.349593 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb9d78e-cbb0-4ba2-9b87-bddce7216169 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:45.354997 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a14aa-1c36-4cbc-a047-8dddf88fd43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:45.355636 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a14aa-1c36-4cbc-a047-8dddf88fd43c admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:45.355913 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a14aa-1c36-4cbc-a047-8dddf88fd43c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:45.356244 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a14aa-1c36-4cbc-a047-8dddf88fd43c admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:45.356813 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a14aa-1c36-4cbc-a047-8dddf88fd43c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:45.358726 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a14aa-1c36-4cbc-a047-8dddf88fd43c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:45.358833 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ab287-9582-412d-aa8f-8d66743603fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:45.359531 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ab287-9582-412d-aa8f-8d66743603fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:45.360349 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ab287-9582-412d-aa8f-8d66743603fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:45.360349 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ab287-9582-412d-aa8f-8d66743603fa admin admin] PATH_INFO: `/v3/projects/ff0bc2336bdd4b729f4b391e64e876b2/users/551300f824124d49b6e16bd932abe79a/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:45.384074 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ab287-9582-412d-aa8f-8d66743603fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff0bc2336bdd4b729f4b391e64e876b2, user_id=551300f824124d49b6e16bd932abe79a, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:45.389112 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 821/1647] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:10:45 2022] POST /identity/v3/users => generated 285 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:45.389420 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ab287-9582-412d-aa8f-8d66743603fa admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:45.391378 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a14aa-1c36-4cbc-a047-8dddf88fd43c admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:45.404528 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 827/1648] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:10:45 2022] PUT /identity/v3/projects/ff0bc2336bdd4b729f4b391e64e876b2/users/551300f824124d49b6e16bd932abe79a/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:10:45.411378 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a14aa-1c36-4cbc-a047-8dddf88fd43c admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:45.415133 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351d4d8-0f73-4d56-ab09-eef0b6995343 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:45.415423 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351d4d8-0f73-4d56-ab09-eef0b6995343 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:45.415693 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351d4d8-0f73-4d56-ab09-eef0b6995343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:45.451963 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:45.467152 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-4351d4d8-0f73-4d56-ab09-eef0b6995343 None None] MFA Rules not processed for user `551300f824124d49b6e16bd932abe79a`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:45.469883 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-4351d4d8-0f73-4d56-ab09-eef0b6995343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:45.489474 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b816d179-6c59-41e3-b833-7b305a8d8258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:45.490159 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816d179-6c59-41e3-b833-7b305a8d8258 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:45.490461 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816d179-6c59-41e3-b833-7b305a8d8258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:45.490836 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816d179-6c59-41e3-b833-7b305a8d8258 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:45.491530 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b816d179-6c59-41e3-b833-7b305a8d8258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:45.493802 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b816d179-6c59-41e3-b833-7b305a8d8258 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:45.505977 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 822/1649] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:45 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:45.509705 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b816d179-6c59-41e3-b833-7b305a8d8258 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:45.544332 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b816d179-6c59-41e3-b833-7b305a8d8258 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:45.550956 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa98019-a638-4948-a5d4-8466a6de0155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:45.551771 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa98019-a638-4948-a5d4-8466a6de0155 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:45.552318 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa98019-a638-4948-a5d4-8466a6de0155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:45.552915 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa98019-a638-4948-a5d4-8466a6de0155 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:45.555299 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa98019-a638-4948-a5d4-8466a6de0155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:45.557734 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa98019-a638-4948-a5d4-8466a6de0155 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:45.576900 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 828/1650] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:45.577423 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 823/1651] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:45 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:45.584209 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351d4d8-0f73-4d56-ab09-eef0b6995343 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:45.589156 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa98019-a638-4948-a5d4-8466a6de0155 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:45.591964 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-caa98019-a638-4948-a5d4-8466a6de0155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:45.596855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351d4d8-0f73-4d56-ab09-eef0b6995343 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:45.601999 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a2d222-31a1-4020-9ee5-d5c1c9be0119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:45.602675 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2d222-31a1-4020-9ee5-d5c1c9be0119 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:45.603263 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2d222-31a1-4020-9ee5-d5c1c9be0119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:45.603263 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a2d222-31a1-4020-9ee5-d5c1c9be0119 admin admin] PATH_INFO: `/v3/projects/949fb16df8c646dfac8994a783c53702/users/6cdb4c2180214225a7dad23579ffcd0d/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:45.624035 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a2d222-31a1-4020-9ee5-d5c1c9be0119 admin admin] RBAC: Authorizing `identity:create_grant(project_id=949fb16df8c646dfac8994a783c53702, user_id=6cdb4c2180214225a7dad23579ffcd0d, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:45.626980 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a2d222-31a1-4020-9ee5-d5c1c9be0119 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:45.644280 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 829/1652] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:10:45 2022] PUT /identity/v3/projects/949fb16df8c646dfac8994a783c53702/users/6cdb4c2180214225a7dad23579ffcd0d/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:45.656837 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724474ec-e7cb-47a1-b71a-7efebbfe18f8 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:45.657512 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724474ec-e7cb-47a1-b71a-7efebbfe18f8 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:45.657512 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724474ec-e7cb-47a1-b71a-7efebbfe18f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:45.661369 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa98019-a638-4948-a5d4-8466a6de0155 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:45.672721 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:45.692730 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-724474ec-e7cb-47a1-b71a-7efebbfe18f8 None None] MFA Rules not processed for user `6cdb4c2180214225a7dad23579ffcd0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:45.693764 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-724474ec-e7cb-47a1-b71a-7efebbfe18f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:45.719836 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f9eb3-1b79-456f-8901-953d469fe0f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:45.720448 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f9eb3-1b79-456f-8901-953d469fe0f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:45.720726 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f9eb3-1b79-456f-8901-953d469fe0f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:45.720995 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f9eb3-1b79-456f-8901-953d469fe0f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:45.781070 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 830/1653] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:10:45.794195 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724474ec-e7cb-47a1-b71a-7efebbfe18f8 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:45.795934 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-724474ec-e7cb-47a1-b71a-7efebbfe18f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:45.817170 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99f9eb3-1b79-456f-8901-953d469fe0f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:45.818629 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99f9eb3-1b79-456f-8901-953d469fe0f9 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:45.822517 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724474ec-e7cb-47a1-b71a-7efebbfe18f8 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:45.828343 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f336fd1c-67d0-4245-860a-f6c5a090cc38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:45.829551 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336fd1c-67d0-4245-860a-f6c5a090cc38 service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:45.829835 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336fd1c-67d0-4245-860a-f6c5a090cc38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:45.830288 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 824/1654] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 13 13:10:45.830550 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336fd1c-67d0-4245-860a-f6c5a090cc38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:45.844985 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f336fd1c-67d0-4245-860a-f6c5a090cc38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:45.846533 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f336fd1c-67d0-4245-860a-f6c5a090cc38 service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:45.858985 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 831/1655] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:45 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 13 13:10:45.890068 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f9eb3-1b79-456f-8901-953d469fe0f9 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:45.955426 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:46.025523 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f9eb3-1b79-456f-8901-953d469fe0f9 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:46.054059 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa254ee4-f172-45a5-865b-039a7cd7773a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:46.056428 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa254ee4-f172-45a5-865b-039a7cd7773a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:46.057179 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa254ee4-f172-45a5-865b-039a7cd7773a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:46.057621 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa254ee4-f172-45a5-865b-039a7cd7773a admin admin] PATH_INFO: `/v3/users/6cdb4c2180214225a7dad23579ffcd0d` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:46.075278 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa254ee4-f172-45a5-865b-039a7cd7773a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cdb4c2180214225a7dad23579ffcd0d)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:46.093833 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa254ee4-f172-45a5-865b-039a7cd7773a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:46.188271 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-aa254ee4-f172-45a5-865b-039a7cd7773a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cdb4c2180214225a7dad23579ffcd0d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:46.221662 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-aa254ee4-f172-45a5-865b-039a7cd7773a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cdb4c2180214225a7dad23579ffcd0d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:46.223953 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-aa254ee4-f172-45a5-865b-039a7cd7773a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cdb4c2180214225a7dad23579ffcd0d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:46.247830 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-aa254ee4-f172-45a5-865b-039a7cd7773a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cdb4c2180214225a7dad23579ffcd0d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:46.274871 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-aa254ee4-f172-45a5-865b-039a7cd7773a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cdb4c2180214225a7dad23579ffcd0d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:46.288637 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 825/1656] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:10:45 2022] DELETE /identity/v3/users/6cdb4c2180214225a7dad23579ffcd0d => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:46.294984 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd94d99-23d0-46fd-9744-e7d95aa5cfe0 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:46.295557 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd94d99-23d0-46fd-9744-e7d95aa5cfe0 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:46.295835 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd94d99-23d0-46fd-9744-e7d95aa5cfe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:46.332141 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-0bd94d99-23d0-46fd-9744-e7d95aa5cfe0 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:46.334383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-0bd94d99-23d0-46fd-9744-e7d95aa5cfe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:46.552423 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 832/1657] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:46.565882 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa254ee4-f172-45a5-865b-039a7cd7773a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:46.568143 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-aa254ee4-f172-45a5-865b-039a7cd7773a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:46.670657 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa254ee4-f172-45a5-865b-039a7cd7773a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:46.680045 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd24ce8-2210-49b7-a0af-562c03d74ea2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:46.681080 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd24ce8-2210-49b7-a0af-562c03d74ea2 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:46.681676 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd24ce8-2210-49b7-a0af-562c03d74ea2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:46.682090 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd24ce8-2210-49b7-a0af-562c03d74ea2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:46.695700 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd24ce8-2210-49b7-a0af-562c03d74ea2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:46.697356 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd24ce8-2210-49b7-a0af-562c03d74ea2 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:46.729668 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 826/1658] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:10:47.128090 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd94d99-23d0-46fd-9744-e7d95aa5cfe0 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:47.129685 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-0bd94d99-23d0-46fd-9744-e7d95aa5cfe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:47.147665 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd94d99-23d0-46fd-9744-e7d95aa5cfe0 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:47.153016 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413841b-9a1e-4a6a-a471-cc33e51552b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:47.153894 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413841b-9a1e-4a6a-a471-cc33e51552b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:47.154433 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413841b-9a1e-4a6a-a471-cc33e51552b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:47.154825 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413841b-9a1e-4a6a-a471-cc33e51552b3 admin admin] PATH_INFO: `/v3/projects/949fb16df8c646dfac8994a783c53702` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:47.156911 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413841b-9a1e-4a6a-a471-cc33e51552b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=949fb16df8c646dfac8994a783c53702)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:47.159047 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413841b-9a1e-4a6a-a471-cc33e51552b3 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:47.212294 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-3413841b-9a1e-4a6a-a471-cc33e51552b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '949fb16df8c646dfac8994a783c53702', 'request_id': 'req-3413841b-9a1e-4a6a-a471-cc33e51552b3'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:47.220126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-3413841b-9a1e-4a6a-a471-cc33e51552b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '949fb16df8c646dfac8994a783c53702', 'request_id': 'req-3413841b-9a1e-4a6a-a471-cc33e51552b3'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:47.226323 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-3413841b-9a1e-4a6a-a471-cc33e51552b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '949fb16df8c646dfac8994a783c53702', 'request_id': 'req-3413841b-9a1e-4a6a-a471-cc33e51552b3'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:47.244415 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-3413841b-9a1e-4a6a-a471-cc33e51552b3 admin admin] The token cache is being invalidate because project 949fb16df8c646dfac8994a783c53702 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:10:47.245040 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-3413841b-9a1e-4a6a-a471-cc33e51552b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:47.245752 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-3413841b-9a1e-4a6a-a471-cc33e51552b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:47.250254 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 833/1659] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:10:47 2022] DELETE /identity/v3/projects/949fb16df8c646dfac8994a783c53702 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:47.253634 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57badc-a284-41fb-bf50-1e41df4af820 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:47.254899 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57badc-a284-41fb-bf50-1e41df4af820 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:47.255299 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57badc-a284-41fb-bf50-1e41df4af820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:47.276774 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0b57badc-a284-41fb-bf50-1e41df4af820 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:47.278093 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0b57badc-a284-41fb-bf50-1e41df4af820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:47.327671 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:47.359095 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 827/1660] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:47.363754 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b722e8-fbbb-4dbe-b197-14dec360d5ff None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:47.364050 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b722e8-fbbb-4dbe-b197-14dec360d5ff None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:47.364330 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b722e8-fbbb-4dbe-b197-14dec360d5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:47.387824 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:47.400745 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-87b722e8-fbbb-4dbe-b197-14dec360d5ff None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:47.405077 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-87b722e8-fbbb-4dbe-b197-14dec360d5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:47.420637 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 834/1661] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:10:47.424817 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b57badc-a284-41fb-bf50-1e41df4af820 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:47.435295 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b57badc-a284-41fb-bf50-1e41df4af820 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:47.441416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ddcc38-c560-44b8-80f4-a3057a81262a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:47.442770 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ddcc38-c560-44b8-80f4-a3057a81262a admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:47.442770 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ddcc38-c560-44b8-80f4-a3057a81262a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:47.443107 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ddcc38-c560-44b8-80f4-a3057a81262a admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:47.444324 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ddcc38-c560-44b8-80f4-a3057a81262a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:10:47.444324 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ddcc38-c560-44b8-80f4-a3057a81262a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:47.446646 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ddcc38-c560-44b8-80f4-a3057a81262a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:47.466868 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 828/1662] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:10:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:10:47.468245 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b722e8-fbbb-4dbe-b197-14dec360d5ff None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:47.483639 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b722e8-fbbb-4dbe-b197-14dec360d5ff None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:47.489943 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6a729-f268-4e48-b7d3-38493cb101ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:47.490563 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6a729-f268-4e48-b7d3-38493cb101ae admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:47.490845 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6a729-f268-4e48-b7d3-38493cb101ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:47.491148 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6a729-f268-4e48-b7d3-38493cb101ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:47.491854 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a6a729-f268-4e48-b7d3-38493cb101ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:47.493887 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a6a729-f268-4e48-b7d3-38493cb101ae admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:47.539571 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 835/1663] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:10:47 2022] POST /identity/v3/projects => generated 379 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:47.540252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ddcc38-c560-44b8-80f4-a3057a81262a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:47.555729 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ddcc38-c560-44b8-80f4-a3057a81262a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:47.621067 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48af40a1-1f78-4351-a5af-c65ebaa59506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:47.621710 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af40a1-1f78-4351-a5af-c65ebaa59506 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:47.621942 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af40a1-1f78-4351-a5af-c65ebaa59506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:47.622215 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af40a1-1f78-4351-a5af-c65ebaa59506 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:47.622855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48af40a1-1f78-4351-a5af-c65ebaa59506 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:47.624882 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48af40a1-1f78-4351-a5af-c65ebaa59506 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:47.652661 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 829/1664] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:10:47 2022] POST /identity/v3/users => generated 290 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:47.653915 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6a729-f268-4e48-b7d3-38493cb101ae admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:47.670223 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6a729-f268-4e48-b7d3-38493cb101ae admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:47.675424 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258886a7-daf7-4971-b271-b6f8a898002f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:47.676053 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258886a7-daf7-4971-b271-b6f8a898002f admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:47.676335 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258886a7-daf7-4971-b271-b6f8a898002f admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:47.676610 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258886a7-daf7-4971-b271-b6f8a898002f admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:47.678753 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258886a7-daf7-4971-b271-b6f8a898002f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:47.679612 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258886a7-daf7-4971-b271-b6f8a898002f admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:47.693289 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 836/1665] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:47.696670 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48af40a1-1f78-4351-a5af-c65ebaa59506 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:47.715034 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48af40a1-1f78-4351-a5af-c65ebaa59506 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:47.730331 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fbc3e8-5d51-4ab4-9a29-48e978891b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:47.730965 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbc3e8-5d51-4ab4-9a29-48e978891b28 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:47.731292 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbc3e8-5d51-4ab4-9a29-48e978891b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:47.731568 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbc3e8-5d51-4ab4-9a29-48e978891b28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:47.732275 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fbc3e8-5d51-4ab4-9a29-48e978891b28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:47.733928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fbc3e8-5d51-4ab4-9a29-48e978891b28 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:47.746327 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 830/1666] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:47.750619 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258886a7-daf7-4971-b271-b6f8a898002f admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:47.761298 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258886a7-daf7-4971-b271-b6f8a898002f admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:47.766686 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a6067-699b-4e70-93d6-22504c475c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:47.767598 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a6067-699b-4e70-93d6-22504c475c36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:47.767888 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a6067-699b-4e70-93d6-22504c475c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:47.768166 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a6067-699b-4e70-93d6-22504c475c36 admin admin] PATH_INFO: `/v3/projects/f85144e13e254365a86aa785093c8575/users/8569afc405f24a0993a023c818a0c272/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:47.786614 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a6067-699b-4e70-93d6-22504c475c36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f85144e13e254365a86aa785093c8575, user_id=8569afc405f24a0993a023c818a0c272, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:47.791558 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a6067-699b-4e70-93d6-22504c475c36 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:47.811140 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 837/1667] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:10:47 2022] PUT /identity/v3/projects/f85144e13e254365a86aa785093c8575/users/8569afc405f24a0993a023c818a0c272/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:47.813728 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529de62-f6a8-414f-b823-613b8f702b18 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:47.814015 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529de62-f6a8-414f-b823-613b8f702b18 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:47.814287 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529de62-f6a8-414f-b823-613b8f702b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:47.839489 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-5529de62-f6a8-414f-b823-613b8f702b18 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:47.840824 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5529de62-f6a8-414f-b823-613b8f702b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:47.914196 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 831/1668] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:47.937767 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a6067-699b-4e70-93d6-22504c475c36 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:47.939675 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-f77a6067-699b-4e70-93d6-22504c475c36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:48.047234 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a6067-699b-4e70-93d6-22504c475c36 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:48.050450 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b2904-ad29-4bce-8ee6-90d6305aed70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:48.051302 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b2904-ad29-4bce-8ee6-90d6305aed70 service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:48.051751 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b2904-ad29-4bce-8ee6-90d6305aed70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:48.052436 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b2904-ad29-4bce-8ee6-90d6305aed70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:48.064903 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92b2904-ad29-4bce-8ee6-90d6305aed70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:48.066402 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92b2904-ad29-4bce-8ee6-90d6305aed70 service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:48.078447 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 838/1669] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:10:48.740020 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf919e0-0aae-4af3-afc8-b971dddeb6d8 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:48.740490 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf919e0-0aae-4af3-afc8-b971dddeb6d8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:48.740580 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf919e0-0aae-4af3-afc8-b971dddeb6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:48.814183 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0bf919e0-0aae-4af3-afc8-b971dddeb6d8 None None] MFA Rules not processed for user `8569afc405f24a0993a023c818a0c272`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:48.815571 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0bf919e0-0aae-4af3-afc8-b971dddeb6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:48.870392 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:48.900296 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 832/1670] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:48.912422 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b2904-ad29-4bce-8ee6-90d6305aed70 service neutron] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:48.915906 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-c92b2904-ad29-4bce-8ee6-90d6305aed70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:48.987785 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b2904-ad29-4bce-8ee6-90d6305aed70 service neutron] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:48.991465 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4129e-c99b-4524-a284-62c3a0132dda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:48.992110 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4129e-c99b-4524-a284-62c3a0132dda service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:48.992391 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4129e-c99b-4524-a284-62c3a0132dda service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:48.994086 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4129e-c99b-4524-a284-62c3a0132dda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:49.003465 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a4129e-c99b-4524-a284-62c3a0132dda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:49.005283 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a4129e-c99b-4524-a284-62c3a0132dda service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:49.017366 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 839/1671] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:10:50.465366 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2051f36-b770-41b9-8626-808819aec4a1 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:50.465924 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2051f36-b770-41b9-8626-808819aec4a1 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:50.465924 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2051f36-b770-41b9-8626-808819aec4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:50.485890 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-e2051f36-b770-41b9-8626-808819aec4a1 None None] MFA Rules not processed for user `8569afc405f24a0993a023c818a0c272`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:50.487879 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e2051f36-b770-41b9-8626-808819aec4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:50.495307 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 833/1672] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:53.033284 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4129e-c99b-4524-a284-62c3a0132dda service cinder] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:53.034848 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-f3a4129e-c99b-4524-a284-62c3a0132dda service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:53.049113 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4129e-c99b-4524-a284-62c3a0132dda service cinder] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:53.055048 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdaa793-f49f-46a8-a020-4d8ec713f4e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:53.055679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdaa793-f49f-46a8-a020-4d8ec713f4e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:53.056036 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdaa793-f49f-46a8-a020-4d8ec713f4e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:53.056317 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdaa793-f49f-46a8-a020-4d8ec713f4e6 admin admin] PATH_INFO: `/v3/users/e08e29ebce1c4559acad7b3111802070` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:53.058295 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdaa793-f49f-46a8-a020-4d8ec713f4e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e08e29ebce1c4559acad7b3111802070)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:53.060354 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdaa793-f49f-46a8-a020-4d8ec713f4e6 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:53.144683 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:53.151886 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-2bdaa793-f49f-46a8-a020-4d8ec713f4e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e08e29ebce1c4559acad7b3111802070'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:53.171352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-2bdaa793-f49f-46a8-a020-4d8ec713f4e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e08e29ebce1c4559acad7b3111802070'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:53.171352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-2bdaa793-f49f-46a8-a020-4d8ec713f4e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e08e29ebce1c4559acad7b3111802070'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:53.171352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-2bdaa793-f49f-46a8-a020-4d8ec713f4e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e08e29ebce1c4559acad7b3111802070'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:53.177655 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-2bdaa793-f49f-46a8-a020-4d8ec713f4e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e08e29ebce1c4559acad7b3111802070'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:53.200908 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 840/1673] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:10:53 2022] DELETE /identity/v3/users/e08e29ebce1c4559acad7b3111802070 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:53.205183 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ea529-639a-4c8d-a7c5-8776bce50279 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:53.205640 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ea529-639a-4c8d-a7c5-8776bce50279 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:53.205799 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ea529-639a-4c8d-a7c5-8776bce50279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:53.226431 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-e35ea529-639a-4c8d-a7c5-8776bce50279 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:53.227679 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e35ea529-639a-4c8d-a7c5-8776bce50279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:53.316971 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 834/1674] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:53.328618 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdaa793-f49f-46a8-a020-4d8ec713f4e6 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:53.330708 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-2bdaa793-f49f-46a8-a020-4d8ec713f4e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:53.496958 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdaa793-f49f-46a8-a020-4d8ec713f4e6 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:53.500671 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfbe676-a20e-4658-a872-1dfaa1bdd006 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:53.501420 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbe676-a20e-4658-a872-1dfaa1bdd006 service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:53.501885 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbe676-a20e-4658-a872-1dfaa1bdd006 service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:53.502252 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbe676-a20e-4658-a872-1dfaa1bdd006 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:53.514216 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfbe676-a20e-4658-a872-1dfaa1bdd006 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:53.515980 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfbe676-a20e-4658-a872-1dfaa1bdd006 service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:53.527506 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 841/1675] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:10:53.633485 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ea529-639a-4c8d-a7c5-8776bce50279 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:53.635490 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e35ea529-639a-4c8d-a7c5-8776bce50279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:53.654378 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ea529-639a-4c8d-a7c5-8776bce50279 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:53.659980 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638150ca-5a6b-436f-8956-8f2906bd8798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:53.660884 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638150ca-5a6b-436f-8956-8f2906bd8798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:53.661219 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638150ca-5a6b-436f-8956-8f2906bd8798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:53.661500 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638150ca-5a6b-436f-8956-8f2906bd8798 admin admin] PATH_INFO: `/v3/users/bc7dfc9d2f5b405286aaf3ed23a14322` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:53.663658 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638150ca-5a6b-436f-8956-8f2906bd8798 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc7dfc9d2f5b405286aaf3ed23a14322)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:53.665511 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638150ca-5a6b-436f-8956-8f2906bd8798 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:53.742419 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-638150ca-5a6b-436f-8956-8f2906bd8798 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc7dfc9d2f5b405286aaf3ed23a14322'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:53.751829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-638150ca-5a6b-436f-8956-8f2906bd8798 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc7dfc9d2f5b405286aaf3ed23a14322'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:53.752751 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-638150ca-5a6b-436f-8956-8f2906bd8798 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc7dfc9d2f5b405286aaf3ed23a14322'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:53.764702 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-638150ca-5a6b-436f-8956-8f2906bd8798 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc7dfc9d2f5b405286aaf3ed23a14322'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:53.780565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-638150ca-5a6b-436f-8956-8f2906bd8798 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc7dfc9d2f5b405286aaf3ed23a14322'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:53.787555 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 835/1676] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:10:53 2022] DELETE /identity/v3/users/bc7dfc9d2f5b405286aaf3ed23a14322 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:53.792268 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042e1cf-4a56-486a-868a-ca1862a79a69 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:53.792885 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042e1cf-4a56-486a-868a-ca1862a79a69 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:53.793611 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042e1cf-4a56-486a-868a-ca1862a79a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:53.814021 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-5042e1cf-4a56-486a-868a-ca1862a79a69 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:53.815348 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-5042e1cf-4a56-486a-868a-ca1862a79a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:53.877458 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638150ca-5a6b-436f-8956-8f2906bd8798 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:53.880824 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-638150ca-5a6b-436f-8956-8f2906bd8798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:53.887102 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 842/1677] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:53.901677 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042e1cf-4a56-486a-868a-ca1862a79a69 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:53.901834 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638150ca-5a6b-436f-8956-8f2906bd8798 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:53.904267 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-5042e1cf-4a56-486a-868a-ca1862a79a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:53.909955 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd21bbca-c578-4011-9943-c936d659d615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:53.910748 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21bbca-c578-4011-9943-c936d659d615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:53.911191 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21bbca-c578-4011-9943-c936d659d615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:53.911635 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21bbca-c578-4011-9943-c936d659d615 admin admin] PATH_INFO: `/v3/projects/238de2aafc0048f3b538d4624293d6bb` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:53.913971 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd21bbca-c578-4011-9943-c936d659d615 admin admin] RBAC: Authorizing `identity:delete_project(project_id=238de2aafc0048f3b538d4624293d6bb)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:53.917019 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd21bbca-c578-4011-9943-c936d659d615 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:53.984026 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042e1cf-4a56-486a-868a-ca1862a79a69 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:53.998550 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:54.001007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-dd21bbca-c578-4011-9943-c936d659d615 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '238de2aafc0048f3b538d4624293d6bb', 'request_id': 'req-dd21bbca-c578-4011-9943-c936d659d615'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:54.005638 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:54.014608 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-dd21bbca-c578-4011-9943-c936d659d615 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '238de2aafc0048f3b538d4624293d6bb', 'request_id': 'req-dd21bbca-c578-4011-9943-c936d659d615'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:54.024955 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-dd21bbca-c578-4011-9943-c936d659d615 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '238de2aafc0048f3b538d4624293d6bb', 'request_id': 'req-dd21bbca-c578-4011-9943-c936d659d615'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:54.044773 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3bd278-a1e3-4753-adaa-990e451622f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:54.045323 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3bd278-a1e3-4753-adaa-990e451622f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:54.045686 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3bd278-a1e3-4753-adaa-990e451622f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:54.045968 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3bd278-a1e3-4753-adaa-990e451622f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:54.062531 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-dd21bbca-c578-4011-9943-c936d659d615 admin admin] The token cache is being invalidate because project 238de2aafc0048f3b538d4624293d6bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:10:54.062829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-dd21bbca-c578-4011-9943-c936d659d615 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:54.065256 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-dd21bbca-c578-4011-9943-c936d659d615 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:54.071114 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 836/1678] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:10:53 2022] DELETE /identity/v3/projects/238de2aafc0048f3b538d4624293d6bb => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:54.074361 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571f284-40e5-4cb6-90e0-f503d7701c7d None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:54.074857 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571f284-40e5-4cb6-90e0-f503d7701c7d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:54.075303 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571f284-40e5-4cb6-90e0-f503d7701c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:54.102494 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d571f284-40e5-4cb6-90e0-f503d7701c7d None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:54.103791 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d571f284-40e5-4cb6-90e0-f503d7701c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:54.140253 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3bd278-a1e3-4753-adaa-990e451622f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:54.142025 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3bd278-a1e3-4753-adaa-990e451622f9 service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:54.155048 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 843/1679] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 13 13:10:54.189538 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 837/1680] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:54.212642 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73305b65-bf3f-45c4-a5f0-950caba99de7 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:54.213288 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73305b65-bf3f-45c4-a5f0-950caba99de7 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:54.213654 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73305b65-bf3f-45c4-a5f0-950caba99de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:54.238326 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-73305b65-bf3f-45c4-a5f0-950caba99de7 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:54.239945 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-73305b65-bf3f-45c4-a5f0-950caba99de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:54.251314 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 844/1681] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:54.255215 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d571f284-40e5-4cb6-90e0-f503d7701c7d None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:54.268498 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d571f284-40e5-4cb6-90e0-f503d7701c7d None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:54.276091 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d369d5d-0782-4d33-9f01-b216fe0eeaf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:54.276914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d369d5d-0782-4d33-9f01-b216fe0eeaf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:54.277444 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d369d5d-0782-4d33-9f01-b216fe0eeaf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:54.278028 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d369d5d-0782-4d33-9f01-b216fe0eeaf7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:54.279040 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d369d5d-0782-4d33-9f01-b216fe0eeaf7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:10:54.279579 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d369d5d-0782-4d33-9f01-b216fe0eeaf7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:54.282224 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d369d5d-0782-4d33-9f01-b216fe0eeaf7 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:54.298658 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 838/1682] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:10:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:10:54.304093 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73305b65-bf3f-45c4-a5f0-950caba99de7 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:54.315503 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73305b65-bf3f-45c4-a5f0-950caba99de7 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:54.323383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01f58c-52e7-4fbb-aaa0-c4ee41c6d2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:54.325124 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01f58c-52e7-4fbb-aaa0-c4ee41c6d2e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:54.328082 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01f58c-52e7-4fbb-aaa0-c4ee41c6d2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:54.328769 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01f58c-52e7-4fbb-aaa0-c4ee41c6d2e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:54.329656 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff01f58c-52e7-4fbb-aaa0-c4ee41c6d2e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:54.333921 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff01f58c-52e7-4fbb-aaa0-c4ee41c6d2e6 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:54.414579 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 845/1683] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:10:54 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:54.414946 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d369d5d-0782-4d33-9f01-b216fe0eeaf7 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:54.425494 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d369d5d-0782-4d33-9f01-b216fe0eeaf7 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:54.500970 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf25a6-2451-41f9-b6cc-c1b01f470401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:54.501752 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf25a6-2451-41f9-b6cc-c1b01f470401 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:54.502033 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf25a6-2451-41f9-b6cc-c1b01f470401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:54.502322 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf25a6-2451-41f9-b6cc-c1b01f470401 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:54.502936 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdf25a6-2451-41f9-b6cc-c1b01f470401 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:54.505179 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdf25a6-2451-41f9-b6cc-c1b01f470401 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:54.536323 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 839/1684] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:10:54 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:10:54.537315 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01f58c-52e7-4fbb-aaa0-c4ee41c6d2e6 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:54.553164 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01f58c-52e7-4fbb-aaa0-c4ee41c6d2e6 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:54.555384 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf25a6-2451-41f9-b6cc-c1b01f470401 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:54.558543 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1fdf25a6-2451-41f9-b6cc-c1b01f470401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:54.582491 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf25a6-2451-41f9-b6cc-c1b01f470401 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:54.584262 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aed00b-0922-47f3-b91f-357135eeef4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:54.584528 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aed00b-0922-47f3-b91f-357135eeef4b admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:54.584809 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aed00b-0922-47f3-b91f-357135eeef4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:54.585082 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aed00b-0922-47f3-b91f-357135eeef4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:54.585865 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aed00b-0922-47f3-b91f-357135eeef4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:54.588339 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aed00b-0922-47f3-b91f-357135eeef4b admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:54.605053 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 846/1685] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:54 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:54.605694 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281145d-d09c-479b-8ddc-bae25ea08db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:54.606046 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aed00b-0922-47f3-b91f-357135eeef4b admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:54.606705 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281145d-d09c-479b-8ddc-bae25ea08db2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:54.607181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281145d-d09c-479b-8ddc-bae25ea08db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:54.607729 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281145d-d09c-479b-8ddc-bae25ea08db2 admin admin] PATH_INFO: `/v3/projects/09f0359d0d60455ab08e2fd479a97cd8` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:54.618541 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281145d-d09c-479b-8ddc-bae25ea08db2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09f0359d0d60455ab08e2fd479a97cd8)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:54.618744 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aed00b-0922-47f3-b91f-357135eeef4b admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:54.621863 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281145d-d09c-479b-8ddc-bae25ea08db2 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:54.631526 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa7e4d-4ff3-4f40-a753-10cba73e5855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:54.632340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa7e4d-4ff3-4f40-a753-10cba73e5855 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:54.633010 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa7e4d-4ff3-4f40-a753-10cba73e5855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:54.633615 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa7e4d-4ff3-4f40-a753-10cba73e5855 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:54.634103 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa7e4d-4ff3-4f40-a753-10cba73e5855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:54.642058 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa7e4d-4ff3-4f40-a753-10cba73e5855 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:54.654693 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 847/1686] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:54 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:54.659246 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa7e4d-4ff3-4f40-a753-10cba73e5855 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:54.675352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa7e4d-4ff3-4f40-a753-10cba73e5855 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:54.688355 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:54.692184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3281145d-d09c-479b-8ddc-bae25ea08db2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09f0359d0d60455ab08e2fd479a97cd8', 'request_id': 'req-3281145d-d09c-479b-8ddc-bae25ea08db2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:54.701441 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3281145d-d09c-479b-8ddc-bae25ea08db2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09f0359d0d60455ab08e2fd479a97cd8', 'request_id': 'req-3281145d-d09c-479b-8ddc-bae25ea08db2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:54.711684 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3281145d-d09c-479b-8ddc-bae25ea08db2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09f0359d0d60455ab08e2fd479a97cd8', 'request_id': 'req-3281145d-d09c-479b-8ddc-bae25ea08db2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:54.729298 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3281145d-d09c-479b-8ddc-bae25ea08db2 admin admin] The token cache is being invalidate because project 09f0359d0d60455ab08e2fd479a97cd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:10:54.729753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3281145d-d09c-479b-8ddc-bae25ea08db2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:54.730733 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3281145d-d09c-479b-8ddc-bae25ea08db2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:54.732888 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 840/1687] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:10:54 2022] DELETE /identity/v3/projects/09f0359d0d60455ab08e2fd479a97cd8 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:54.737025 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281145d-d09c-479b-8ddc-bae25ea08db2 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:54.738385 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3281145d-d09c-479b-8ddc-bae25ea08db2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:54.756815 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6611e3b-fd3a-4479-b6cf-c3aeaaaad57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:54.757827 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6611e3b-fd3a-4479-b6cf-c3aeaaaad57a admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:54.758285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6611e3b-fd3a-4479-b6cf-c3aeaaaad57a admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:54.758732 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6611e3b-fd3a-4479-b6cf-c3aeaaaad57a admin admin] PATH_INFO: `/v3/projects/a71a5ac3063d4d3d910e73a24f4d923f/users/02b4ab4d1b1546a88110fe75e7b3bc45/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:54.780252 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6611e3b-fd3a-4479-b6cf-c3aeaaaad57a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a71a5ac3063d4d3d910e73a24f4d923f, user_id=02b4ab4d1b1546a88110fe75e7b3bc45, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:54.784164 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6611e3b-fd3a-4479-b6cf-c3aeaaaad57a admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:54.801166 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 848/1688] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:10:54 2022] PUT /identity/v3/projects/a71a5ac3063d4d3d910e73a24f4d923f/users/02b4ab4d1b1546a88110fe75e7b3bc45/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:54.812279 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aef4ca-ca5a-415b-b8cd-01c92922fa94 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:54.812498 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aef4ca-ca5a-415b-b8cd-01c92922fa94 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:54.812776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aef4ca-ca5a-415b-b8cd-01c92922fa94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:54.820416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:54.845730 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-65aef4ca-ca5a-415b-b8cd-01c92922fa94 None None] MFA Rules not processed for user `02b4ab4d1b1546a88110fe75e7b3bc45`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:54.846947 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-65aef4ca-ca5a-415b-b8cd-01c92922fa94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:54.855277 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281145d-d09c-479b-8ddc-bae25ea08db2 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:54.918303 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 849/1689] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:10:54.934550 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aef4ca-ca5a-415b-b8cd-01c92922fa94 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:54.935951 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-65aef4ca-ca5a-415b-b8cd-01c92922fa94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:54.938482 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f8458-79be-457f-86e0-8ad79c54469b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:54.939272 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f8458-79be-457f-86e0-8ad79c54469b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:54.939613 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f8458-79be-457f-86e0-8ad79c54469b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:54.939960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f8458-79be-457f-86e0-8ad79c54469b admin admin] PATH_INFO: `/v3/users/2bb65b7d7c624e6bb1414020eac87271` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:54.941840 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f8458-79be-457f-86e0-8ad79c54469b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bb65b7d7c624e6bb1414020eac87271)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:54.946223 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f8458-79be-457f-86e0-8ad79c54469b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:54.995634 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cb3f8458-79be-457f-86e0-8ad79c54469b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bb65b7d7c624e6bb1414020eac87271'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:54.997710 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aef4ca-ca5a-415b-b8cd-01c92922fa94 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:55.001486 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac8863a-40b7-4d44-aaea-a20e76f6076d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:55.002100 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8863a-40b7-4d44-aaea-a20e76f6076d service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:55.002381 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8863a-40b7-4d44-aaea-a20e76f6076d service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:55.002657 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8863a-40b7-4d44-aaea-a20e76f6076d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:55.003021 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cb3f8458-79be-457f-86e0-8ad79c54469b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bb65b7d7c624e6bb1414020eac87271'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:55.003703 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cb3f8458-79be-457f-86e0-8ad79c54469b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bb65b7d7c624e6bb1414020eac87271'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:55.011159 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cb3f8458-79be-457f-86e0-8ad79c54469b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bb65b7d7c624e6bb1414020eac87271'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:55.011761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac8863a-40b7-4d44-aaea-a20e76f6076d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:55.013227 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac8863a-40b7-4d44-aaea-a20e76f6076d service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:55.022977 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 850/1690] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 13 13:10:55.024228 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cb3f8458-79be-457f-86e0-8ad79c54469b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bb65b7d7c624e6bb1414020eac87271'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:55.030172 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 841/1691] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:10:54 2022] DELETE /identity/v3/users/2bb65b7d7c624e6bb1414020eac87271 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:10:55.331032 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac8863a-40b7-4d44-aaea-a20e76f6076d service cinder] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:55.349359 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac8863a-40b7-4d44-aaea-a20e76f6076d service cinder] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:55.443444 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75202baa-ceab-45ef-aca7-4ea827875cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:55.444386 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75202baa-ceab-45ef-aca7-4ea827875cc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:55.444676 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75202baa-ceab-45ef-aca7-4ea827875cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:55.445484 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75202baa-ceab-45ef-aca7-4ea827875cc7 admin admin] PATH_INFO: `/v3/projects/b714f9e378a347bbb78cf393f2798dbe` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:55.446757 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75202baa-ceab-45ef-aca7-4ea827875cc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b714f9e378a347bbb78cf393f2798dbe)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:55.448811 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75202baa-ceab-45ef-aca7-4ea827875cc7 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:55.494392 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:55.505021 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-75202baa-ceab-45ef-aca7-4ea827875cc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b714f9e378a347bbb78cf393f2798dbe', 'request_id': 'req-75202baa-ceab-45ef-aca7-4ea827875cc7'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:55.511389 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-75202baa-ceab-45ef-aca7-4ea827875cc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b714f9e378a347bbb78cf393f2798dbe', 'request_id': 'req-75202baa-ceab-45ef-aca7-4ea827875cc7'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:55.517443 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-75202baa-ceab-45ef-aca7-4ea827875cc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b714f9e378a347bbb78cf393f2798dbe', 'request_id': 'req-75202baa-ceab-45ef-aca7-4ea827875cc7'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:55.532383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-75202baa-ceab-45ef-aca7-4ea827875cc7 admin admin] The token cache is being invalidate because project b714f9e378a347bbb78cf393f2798dbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:10:55.532710 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-75202baa-ceab-45ef-aca7-4ea827875cc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:55.533948 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-75202baa-ceab-45ef-aca7-4ea827875cc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:10:55.535420 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 851/1692] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:10:55 2022] DELETE /identity/v3/projects/b714f9e378a347bbb78cf393f2798dbe => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:55.541019 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82eb14-e480-4b1b-bd81-9b431231b118 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:55.541596 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82eb14-e480-4b1b-bd81-9b431231b118 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:55.541596 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82eb14-e480-4b1b-bd81-9b431231b118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:55.549266 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:55.564076 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-1f82eb14-e480-4b1b-bd81-9b431231b118 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:55.565677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1f82eb14-e480-4b1b-bd81-9b431231b118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:55.666345 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 842/1693] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:10:55.674435 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3fd4d-540d-4c64-9b83-7edae952b33e None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:55.674715 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3fd4d-540d-4c64-9b83-7edae952b33e None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:55.674983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3fd4d-540d-4c64-9b83-7edae952b33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:55.699238 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-07c3fd4d-540d-4c64-9b83-7edae952b33e None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:55.700349 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-07c3fd4d-540d-4c64-9b83-7edae952b33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:55.719360 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 852/1694] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:55.719460 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82eb14-e480-4b1b-bd81-9b431231b118 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:55.723353 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82eb14-e480-4b1b-bd81-9b431231b118 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:55.728240 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4da69-41f0-410b-8a81-cc77c23eb5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:55.729652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4da69-41f0-410b-8a81-cc77c23eb5a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:55.729939 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4da69-41f0-410b-8a81-cc77c23eb5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:55.730209 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4da69-41f0-410b-8a81-cc77c23eb5a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:55.730856 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4da69-41f0-410b-8a81-cc77c23eb5a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:10:55.731185 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4da69-41f0-410b-8a81-cc77c23eb5a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:55.732859 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4da69-41f0-410b-8a81-cc77c23eb5a4 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:55.745471 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 843/1695] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:10:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:10:55.750059 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3fd4d-540d-4c64-9b83-7edae952b33e None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:55.759998 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3fd4d-540d-4c64-9b83-7edae952b33e None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:55.764871 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99028f12-8d61-4454-a670-a552d9ce65ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:55.765671 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99028f12-8d61-4454-a670-a552d9ce65ea admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:55.766160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99028f12-8d61-4454-a670-a552d9ce65ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:55.766511 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99028f12-8d61-4454-a670-a552d9ce65ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:55.767388 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99028f12-8d61-4454-a670-a552d9ce65ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:55.769659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99028f12-8d61-4454-a670-a552d9ce65ea admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:55.797157 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 853/1696] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:10:55 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:55.801316 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4da69-41f0-410b-8a81-cc77c23eb5a4 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:55.811780 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4da69-41f0-410b-8a81-cc77c23eb5a4 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:55.869140 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff2d3b-6294-4fc6-b227-fa50463d08ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:55.869747 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff2d3b-6294-4fc6-b227-fa50463d08ad admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:55.870028 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff2d3b-6294-4fc6-b227-fa50463d08ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:55.870296 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff2d3b-6294-4fc6-b227-fa50463d08ad admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:55.870930 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff2d3b-6294-4fc6-b227-fa50463d08ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:55.872860 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff2d3b-6294-4fc6-b227-fa50463d08ad admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:55.897727 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 844/1697] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:10:55 2022] POST /identity/v3/users => generated 285 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:10:55.898921 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99028f12-8d61-4454-a670-a552d9ce65ea admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:55.910972 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99028f12-8d61-4454-a670-a552d9ce65ea admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:55.916329 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23119a43-5827-4bc1-93f5-f139c4eefce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:55.917055 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23119a43-5827-4bc1-93f5-f139c4eefce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:55.917335 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23119a43-5827-4bc1-93f5-f139c4eefce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:55.917607 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23119a43-5827-4bc1-93f5-f139c4eefce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:55.918250 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23119a43-5827-4bc1-93f5-f139c4eefce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:55.919866 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23119a43-5827-4bc1-93f5-f139c4eefce8 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:55.931855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 854/1698] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:55.935857 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff2d3b-6294-4fc6-b227-fa50463d08ad admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:55.946593 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff2d3b-6294-4fc6-b227-fa50463d08ad admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:55.951279 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0eb9b-32c9-47dd-994e-e6432c2faec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:55.951945 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0eb9b-32c9-47dd-994e-e6432c2faec1 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:55.952234 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0eb9b-32c9-47dd-994e-e6432c2faec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:55.952506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0eb9b-32c9-47dd-994e-e6432c2faec1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:55.953144 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc0eb9b-32c9-47dd-994e-e6432c2faec1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:55.954736 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc0eb9b-32c9-47dd-994e-e6432c2faec1 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:55.966045 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 845/1699] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:55.970127 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23119a43-5827-4bc1-93f5-f139c4eefce8 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:55.980100 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23119a43-5827-4bc1-93f5-f139c4eefce8 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:55.986040 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25cb904-8a21-4811-bbfd-ec0afd628ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:55.986749 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25cb904-8a21-4811-bbfd-ec0afd628ff6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:55.990319 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25cb904-8a21-4811-bbfd-ec0afd628ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:55.990612 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25cb904-8a21-4811-bbfd-ec0afd628ff6 admin admin] PATH_INFO: `/v3/projects/874a82c1c4134e9e8576de7036dee9df/users/12b223d2d17d438b9ef39afbba612391/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:56.006673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25cb904-8a21-4811-bbfd-ec0afd628ff6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=874a82c1c4134e9e8576de7036dee9df, user_id=12b223d2d17d438b9ef39afbba612391, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:56.009987 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25cb904-8a21-4811-bbfd-ec0afd628ff6 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:56.022755 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 855/1700] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:10:55 2022] PUT /identity/v3/projects/874a82c1c4134e9e8576de7036dee9df/users/12b223d2d17d438b9ef39afbba612391/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:56.032961 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200e6a9-9c52-4e02-9e97-13cc8b51130e None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:56.033250 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200e6a9-9c52-4e02-9e97-13cc8b51130e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:56.033522 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200e6a9-9c52-4e02-9e97-13cc8b51130e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:56.061913 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b200e6a9-9c52-4e02-9e97-13cc8b51130e None None] MFA Rules not processed for user `12b223d2d17d438b9ef39afbba612391`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:56.063176 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b200e6a9-9c52-4e02-9e97-13cc8b51130e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:56.129695 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:56.133726 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 846/1701] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:56.138095 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25cb904-8a21-4811-bbfd-ec0afd628ff6 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:56.202714 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25cb904-8a21-4811-bbfd-ec0afd628ff6 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:56.207375 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e9823-5411-4222-9b5e-7f5857f6e4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:56.207979 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e9823-5411-4222-9b5e-7f5857f6e4fc admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:56.208260 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e9823-5411-4222-9b5e-7f5857f6e4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:56.208543 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e9823-5411-4222-9b5e-7f5857f6e4fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:56.209168 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e9823-5411-4222-9b5e-7f5857f6e4fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:56.211079 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e9823-5411-4222-9b5e-7f5857f6e4fc admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:56.237182 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 856/1702] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:10:56 2022] POST /identity/v3/projects => generated 367 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:56.241227 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200e6a9-9c52-4e02-9e97-13cc8b51130e None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:56.252418 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200e6a9-9c52-4e02-9e97-13cc8b51130e None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:56.310588 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05a360-1c39-41cb-b319-079d5c0e7117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:56.311236 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05a360-1c39-41cb-b319-079d5c0e7117 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:56.311530 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05a360-1c39-41cb-b319-079d5c0e7117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:56.311789 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05a360-1c39-41cb-b319-079d5c0e7117 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:56.312437 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05a360-1c39-41cb-b319-079d5c0e7117 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:56.314366 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05a360-1c39-41cb-b319-079d5c0e7117 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:56.341769 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 847/1703] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:10:56 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:10:56.346012 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e9823-5411-4222-9b5e-7f5857f6e4fc admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:56.357111 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e9823-5411-4222-9b5e-7f5857f6e4fc admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:56.361831 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e026acc-a66b-4d93-ab45-2c373313c401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:56.362525 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e026acc-a66b-4d93-ab45-2c373313c401 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:56.362705 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e026acc-a66b-4d93-ab45-2c373313c401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:56.362978 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e026acc-a66b-4d93-ab45-2c373313c401 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:56.363652 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e026acc-a66b-4d93-ab45-2c373313c401 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:56.365276 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e026acc-a66b-4d93-ab45-2c373313c401 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:56.376999 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 857/1704] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:56.380938 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05a360-1c39-41cb-b319-079d5c0e7117 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:56.391457 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05a360-1c39-41cb-b319-079d5c0e7117 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:56.396612 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7453a-3b27-477e-969a-d5d0128b8d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:56.397274 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7453a-3b27-477e-969a-d5d0128b8d22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:56.397555 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7453a-3b27-477e-969a-d5d0128b8d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:56.397857 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7453a-3b27-477e-969a-d5d0128b8d22 admin admin] PATH_INFO: `/v3/projects/2b68b7e6080146678c4cc02d17dc2e9f/users/400775e56ba644ed8b86d56cb2d56aea/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:56.414181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7453a-3b27-477e-969a-d5d0128b8d22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b68b7e6080146678c4cc02d17dc2e9f, user_id=400775e56ba644ed8b86d56cb2d56aea, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:56.416910 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7453a-3b27-477e-969a-d5d0128b8d22 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:56.430708 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 848/1705] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:10:56 2022] PUT /identity/v3/projects/2b68b7e6080146678c4cc02d17dc2e9f/users/400775e56ba644ed8b86d56cb2d56aea/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:10:56.434643 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e026acc-a66b-4d93-ab45-2c373313c401 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:56.444704 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e026acc-a66b-4d93-ab45-2c373313c401 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:56.479641 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:10:56.511964 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c1001e-4812-4411-a241-41a13cc240c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:56.512732 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c1001e-4812-4411-a241-41a13cc240c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:56.513341 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c1001e-4812-4411-a241-41a13cc240c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:56.513723 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c1001e-4812-4411-a241-41a13cc240c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:56.514440 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c1001e-4812-4411-a241-41a13cc240c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:56.516436 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c1001e-4812-4411-a241-41a13cc240c4 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:56.527627 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 858/1706] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:10:56 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:10:56.531821 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7453a-3b27-477e-969a-d5d0128b8d22 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:56.545594 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7453a-3b27-477e-969a-d5d0128b8d22 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:56.550882 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aed01a-78d2-4267-808f-c734007bf142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:56.551752 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aed01a-78d2-4267-808f-c734007bf142 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:56.552475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aed01a-78d2-4267-808f-c734007bf142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:56.552818 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aed01a-78d2-4267-808f-c734007bf142 admin admin] PATH_INFO: `/v3/projects/2b68b7e6080146678c4cc02d17dc2e9f/users/400775e56ba644ed8b86d56cb2d56aea/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:56.555246 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aed01a-78d2-4267-808f-c734007bf142 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b68b7e6080146678c4cc02d17dc2e9f, user_id=400775e56ba644ed8b86d56cb2d56aea, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:56.558157 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aed01a-78d2-4267-808f-c734007bf142 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:56.583480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 849/1707] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:10:56 2022] PUT /identity/v3/projects/2b68b7e6080146678c4cc02d17dc2e9f/users/400775e56ba644ed8b86d56cb2d56aea/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:10:56.595722 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f174db4-ec9a-43c9-ac71-cb7071fdf364 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:56.596014 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f174db4-ec9a-43c9-ac71-cb7071fdf364 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:56.596691 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f174db4-ec9a-43c9-ac71-cb7071fdf364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:56.636230 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-9f174db4-ec9a-43c9-ac71-cb7071fdf364 None None] MFA Rules not processed for user `400775e56ba644ed8b86d56cb2d56aea`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:10:56.637503 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-9f174db4-ec9a-43c9-ac71-cb7071fdf364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:56.708311 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 859/1708] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:10:56 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:10:56.719668 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aed01a-78d2-4267-808f-c734007bf142 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:56.721407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a4aed01a-78d2-4267-808f-c734007bf142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:10:56.780926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aed01a-78d2-4267-808f-c734007bf142 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:56.784705 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3002349-39ac-4c0f-a227-f56829c48133 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:56.785314 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3002349-39ac-4c0f-a227-f56829c48133 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:56.785600 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3002349-39ac-4c0f-a227-f56829c48133 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:56.785868 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3002349-39ac-4c0f-a227-f56829c48133 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:56.795334 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3002349-39ac-4c0f-a227-f56829c48133 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:56.796766 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3002349-39ac-4c0f-a227-f56829c48133 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:56.806462 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 850/1709] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:56 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:10:56.851000 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f174db4-ec9a-43c9-ac71-cb7071fdf364 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:10:56.860406 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f174db4-ec9a-43c9-ac71-cb7071fdf364 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:10:56.864065 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4243e314-3bad-4dc7-a284-c4c29d390f51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:10:56.864731 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243e314-3bad-4dc7-a284-c4c29d390f51 service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:10:56.865012 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243e314-3bad-4dc7-a284-c4c29d390f51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:10:56.865277 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243e314-3bad-4dc7-a284-c4c29d390f51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:10:56.927118 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4243e314-3bad-4dc7-a284-c4c29d390f51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:10:56.928571 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4243e314-3bad-4dc7-a284-c4c29d390f51 service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:10:56.937615 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 860/1710] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:10:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:11:00.299465 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3002349-39ac-4c0f-a227-f56829c48133 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:00.350614 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:00.361704 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3002349-39ac-4c0f-a227-f56829c48133 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:00.366328 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8242031c-0fb0-40a1-a61a-e575271183f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:00.367489 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242031c-0fb0-40a1-a61a-e575271183f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:00.367766 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242031c-0fb0-40a1-a61a-e575271183f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:00.368034 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242031c-0fb0-40a1-a61a-e575271183f1 admin admin] PATH_INFO: `/v3/users/12b223d2d17d438b9ef39afbba612391` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:00.369131 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8242031c-0fb0-40a1-a61a-e575271183f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12b223d2d17d438b9ef39afbba612391)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:00.371116 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8242031c-0fb0-40a1-a61a-e575271183f1 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:00.416420 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-8242031c-0fb0-40a1-a61a-e575271183f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12b223d2d17d438b9ef39afbba612391'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:00.423153 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-8242031c-0fb0-40a1-a61a-e575271183f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12b223d2d17d438b9ef39afbba612391'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:00.423811 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-8242031c-0fb0-40a1-a61a-e575271183f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12b223d2d17d438b9ef39afbba612391'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:00.431583 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-8242031c-0fb0-40a1-a61a-e575271183f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12b223d2d17d438b9ef39afbba612391'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:00.443494 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-8242031c-0fb0-40a1-a61a-e575271183f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12b223d2d17d438b9ef39afbba612391'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:00.449822 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 851/1711] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:11:00 2022] DELETE /identity/v3/users/12b223d2d17d438b9ef39afbba612391 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:00.454447 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd90486-9d98-4a1b-8433-e2277b7ea0d0 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:00.454889 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd90486-9d98-4a1b-8433-e2277b7ea0d0 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:00.454993 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd90486-9d98-4a1b-8433-e2277b7ea0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:00.473451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-0cd90486-9d98-4a1b-8433-e2277b7ea0d0 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:00.474640 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-0cd90486-9d98-4a1b-8433-e2277b7ea0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:00.521334 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:00.532985 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 861/1712] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:00.543189 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8242031c-0fb0-40a1-a61a-e575271183f1 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:00.544581 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8242031c-0fb0-40a1-a61a-e575271183f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:00.600546 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8242031c-0fb0-40a1-a61a-e575271183f1 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:00.604743 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59f7b0-cb0a-4d42-a555-951665a2973f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:00.605346 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59f7b0-cb0a-4d42-a555-951665a2973f service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:00.605624 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59f7b0-cb0a-4d42-a555-951665a2973f service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:00.605894 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59f7b0-cb0a-4d42-a555-951665a2973f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:00.614976 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59f7b0-cb0a-4d42-a555-951665a2973f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:00.616440 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc59f7b0-cb0a-4d42-a555-951665a2973f service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:00.626450 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 852/1713] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:11:00.941504 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd90486-9d98-4a1b-8433-e2277b7ea0d0 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:00.943003 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-0cd90486-9d98-4a1b-8433-e2277b7ea0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:00.965160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd90486-9d98-4a1b-8433-e2277b7ea0d0 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:00.985051 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2e149-1797-4cad-9ac9-fe6e079793cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:00.985713 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2e149-1797-4cad-9ac9-fe6e079793cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:00.985995 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2e149-1797-4cad-9ac9-fe6e079793cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:00.986272 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc2e149-1797-4cad-9ac9-fe6e079793cf admin admin] PATH_INFO: `/v3/projects/874a82c1c4134e9e8576de7036dee9df` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:00.990943 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc2e149-1797-4cad-9ac9-fe6e079793cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=874a82c1c4134e9e8576de7036dee9df)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:00.993032 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc2e149-1797-4cad-9ac9-fe6e079793cf admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:01.067441 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-2bc2e149-1797-4cad-9ac9-fe6e079793cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '874a82c1c4134e9e8576de7036dee9df', 'request_id': 'req-2bc2e149-1797-4cad-9ac9-fe6e079793cf'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:01.073865 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-2bc2e149-1797-4cad-9ac9-fe6e079793cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '874a82c1c4134e9e8576de7036dee9df', 'request_id': 'req-2bc2e149-1797-4cad-9ac9-fe6e079793cf'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:01.081195 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-2bc2e149-1797-4cad-9ac9-fe6e079793cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '874a82c1c4134e9e8576de7036dee9df', 'request_id': 'req-2bc2e149-1797-4cad-9ac9-fe6e079793cf'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:01.106482 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-2bc2e149-1797-4cad-9ac9-fe6e079793cf admin admin] The token cache is being invalidate because project 874a82c1c4134e9e8576de7036dee9df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:11:01.106778 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-2bc2e149-1797-4cad-9ac9-fe6e079793cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:01.107514 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-2bc2e149-1797-4cad-9ac9-fe6e079793cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:01.109226 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 862/1714] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:11:00 2022] DELETE /identity/v3/projects/874a82c1c4134e9e8576de7036dee9df => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:01.112997 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59f7b0-cb0a-4d42-a555-951665a2973f service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:01.123779 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59f7b0-cb0a-4d42-a555-951665a2973f service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:01.207643 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c415c-9ac4-44e8-8641-3fc5fc75cbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:01.208308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c415c-9ac4-44e8-8641-3fc5fc75cbad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:01.208640 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c415c-9ac4-44e8-8641-3fc5fc75cbad admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:01.210938 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c415c-9ac4-44e8-8641-3fc5fc75cbad admin admin] PATH_INFO: `/v3/users/400775e56ba644ed8b86d56cb2d56aea` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:01.210938 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c415c-9ac4-44e8-8641-3fc5fc75cbad admin admin] RBAC: Authorizing `identity:delete_user(user_id=400775e56ba644ed8b86d56cb2d56aea)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:01.212760 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c415c-9ac4-44e8-8641-3fc5fc75cbad admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:01.230214 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:01.271013 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0d3c415c-9ac4-44e8-8641-3fc5fc75cbad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '400775e56ba644ed8b86d56cb2d56aea'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:01.280680 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0d3c415c-9ac4-44e8-8641-3fc5fc75cbad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '400775e56ba644ed8b86d56cb2d56aea'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:01.281608 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0d3c415c-9ac4-44e8-8641-3fc5fc75cbad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '400775e56ba644ed8b86d56cb2d56aea'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:01.294402 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0d3c415c-9ac4-44e8-8641-3fc5fc75cbad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '400775e56ba644ed8b86d56cb2d56aea'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:01.316223 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0d3c415c-9ac4-44e8-8641-3fc5fc75cbad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '400775e56ba644ed8b86d56cb2d56aea'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:01.326566 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 853/1715] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:11:01 2022] DELETE /identity/v3/users/400775e56ba644ed8b86d56cb2d56aea => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:11:01.612875 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2e149-1797-4cad-9ac9-fe6e079793cf admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:01.614344 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-2bc2e149-1797-4cad-9ac9-fe6e079793cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:01.631523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc2e149-1797-4cad-9ac9-fe6e079793cf admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:01.689616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e9f12-7690-4732-bb06-f32f10b9a7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:01.690267 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e9f12-7690-4732-bb06-f32f10b9a7d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:01.690545 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e9f12-7690-4732-bb06-f32f10b9a7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:01.690811 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e9f12-7690-4732-bb06-f32f10b9a7d9 admin admin] PATH_INFO: `/v3/projects/2b68b7e6080146678c4cc02d17dc2e9f` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:01.691966 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e9f12-7690-4732-bb06-f32f10b9a7d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b68b7e6080146678c4cc02d17dc2e9f)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:01.693859 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e9f12-7690-4732-bb06-f32f10b9a7d9 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:01.740120 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-352e9f12-7690-4732-bb06-f32f10b9a7d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b68b7e6080146678c4cc02d17dc2e9f', 'request_id': 'req-352e9f12-7690-4732-bb06-f32f10b9a7d9'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:01.746812 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-352e9f12-7690-4732-bb06-f32f10b9a7d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b68b7e6080146678c4cc02d17dc2e9f', 'request_id': 'req-352e9f12-7690-4732-bb06-f32f10b9a7d9'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:01.754057 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-352e9f12-7690-4732-bb06-f32f10b9a7d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b68b7e6080146678c4cc02d17dc2e9f', 'request_id': 'req-352e9f12-7690-4732-bb06-f32f10b9a7d9'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:01.766388 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-352e9f12-7690-4732-bb06-f32f10b9a7d9 admin admin] The token cache is being invalidate because project 2b68b7e6080146678c4cc02d17dc2e9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:11:01.766691 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-352e9f12-7690-4732-bb06-f32f10b9a7d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:01.767381 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-352e9f12-7690-4732-bb06-f32f10b9a7d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:01.768967 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 863/1716] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:11:01 2022] DELETE /identity/v3/projects/2b68b7e6080146678c4cc02d17dc2e9f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:01.774481 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a78f6e-85da-46c0-a29e-245179734d26 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:01.774776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a78f6e-85da-46c0-a29e-245179734d26 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:01.775076 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a78f6e-85da-46c0-a29e-245179734d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:01.794998 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f0a78f6e-85da-46c0-a29e-245179734d26 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:01.796157 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f0a78f6e-85da-46c0-a29e-245179734d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:01.856019 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 854/1717] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:01.863788 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951c733-15ca-4f26-b844-37399abeaf1d None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:01.864151 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951c733-15ca-4f26-b844-37399abeaf1d None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:01.864531 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951c733-15ca-4f26-b844-37399abeaf1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:01.884734 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-a951c733-15ca-4f26-b844-37399abeaf1d None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:01.885928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-a951c733-15ca-4f26-b844-37399abeaf1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:01.893911 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 864/1718] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:01.898701 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a78f6e-85da-46c0-a29e-245179734d26 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:01.909067 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a78f6e-85da-46c0-a29e-245179734d26 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:01.914639 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487530c-d0ec-4333-a5e8-4b9a85276e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:01.915330 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487530c-d0ec-4333-a5e8-4b9a85276e6c admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:01.915615 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487530c-d0ec-4333-a5e8-4b9a85276e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:01.915891 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487530c-d0ec-4333-a5e8-4b9a85276e6c admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:01.916543 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3487530c-d0ec-4333-a5e8-4b9a85276e6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:11:01.916874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3487530c-d0ec-4333-a5e8-4b9a85276e6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:01.918487 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3487530c-d0ec-4333-a5e8-4b9a85276e6c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:01.930886 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 855/1719] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:11:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:11:01.935340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a951c733-15ca-4f26-b844-37399abeaf1d None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:01.939764 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:01.946060 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a951c733-15ca-4f26-b844-37399abeaf1d None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:01.952157 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc748c1-f192-42ee-90e3-b76d33fa4484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:01.952755 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc748c1-f192-42ee-90e3-b76d33fa4484 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:01.953036 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc748c1-f192-42ee-90e3-b76d33fa4484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:01.953310 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc748c1-f192-42ee-90e3-b76d33fa4484 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:01.953955 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc748c1-f192-42ee-90e3-b76d33fa4484 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:01.955901 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc748c1-f192-42ee-90e3-b76d33fa4484 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:01.984859 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 865/1720] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:11:01 2022] POST /identity/v3/projects => generated 393 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:01.989470 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487530c-d0ec-4333-a5e8-4b9a85276e6c admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:01.999415 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487530c-d0ec-4333-a5e8-4b9a85276e6c admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:02.057754 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41779261-03d3-49a4-bb9c-e5907e94e9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:02.058363 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41779261-03d3-49a4-bb9c-e5907e94e9c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:02.058645 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41779261-03d3-49a4-bb9c-e5907e94e9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:02.058917 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41779261-03d3-49a4-bb9c-e5907e94e9c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:02.059583 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41779261-03d3-49a4-bb9c-e5907e94e9c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:02.061494 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41779261-03d3-49a4-bb9c-e5907e94e9c3 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:02.082804 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 856/1721] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:11:01 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:02.087677 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc748c1-f192-42ee-90e3-b76d33fa4484 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:02.097832 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc748c1-f192-42ee-90e3-b76d33fa4484 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:02.102499 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd02edb8-0742-4541-85de-5cfbd5fe1e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:02.103108 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd02edb8-0742-4541-85de-5cfbd5fe1e07 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:02.103388 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd02edb8-0742-4541-85de-5cfbd5fe1e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:02.103659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd02edb8-0742-4541-85de-5cfbd5fe1e07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:02.104298 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd02edb8-0742-4541-85de-5cfbd5fe1e07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:02.105879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd02edb8-0742-4541-85de-5cfbd5fe1e07 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:02.115586 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 866/1722] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:02.119475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41779261-03d3-49a4-bb9c-e5907e94e9c3 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:02.129259 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41779261-03d3-49a4-bb9c-e5907e94e9c3 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:02.134389 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62acaeb1-ff0e-416c-a0dc-006073598810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:02.134980 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62acaeb1-ff0e-416c-a0dc-006073598810 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:02.135299 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62acaeb1-ff0e-416c-a0dc-006073598810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:02.135575 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62acaeb1-ff0e-416c-a0dc-006073598810 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:02.136245 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62acaeb1-ff0e-416c-a0dc-006073598810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:02.137817 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62acaeb1-ff0e-416c-a0dc-006073598810 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:02.143181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:02.149927 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 857/1723] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:02.154012 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd02edb8-0742-4541-85de-5cfbd5fe1e07 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:02.164441 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd02edb8-0742-4541-85de-5cfbd5fe1e07 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:02.168972 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b500ed-35f5-4415-9cf8-2b88f87a9c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:02.169617 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b500ed-35f5-4415-9cf8-2b88f87a9c09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:02.169897 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b500ed-35f5-4415-9cf8-2b88f87a9c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:02.170169 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b500ed-35f5-4415-9cf8-2b88f87a9c09 admin admin] PATH_INFO: `/v3/projects/f2509c4b1f634b1a96668b9c697d9bdf/users/2c3e132b2b2e47599083c951119cac5f/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:02.184366 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b500ed-35f5-4415-9cf8-2b88f87a9c09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2509c4b1f634b1a96668b9c697d9bdf, user_id=2c3e132b2b2e47599083c951119cac5f, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:02.186997 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b500ed-35f5-4415-9cf8-2b88f87a9c09 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:02.199775 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 867/1724] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:02 2022] PUT /identity/v3/projects/f2509c4b1f634b1a96668b9c697d9bdf/users/2c3e132b2b2e47599083c951119cac5f/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:02.210885 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94644a-3c9f-4a95-85b8-0f5c1d622670 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:02.211211 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94644a-3c9f-4a95-85b8-0f5c1d622670 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:02.211486 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94644a-3c9f-4a95-85b8-0f5c1d622670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:02.237868 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-df94644a-3c9f-4a95-85b8-0f5c1d622670 None None] MFA Rules not processed for user `2c3e132b2b2e47599083c951119cac5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:02.239079 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-df94644a-3c9f-4a95-85b8-0f5c1d622670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:02.300173 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 858/1725] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:02.304997 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b500ed-35f5-4415-9cf8-2b88f87a9c09 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:02.366036 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b500ed-35f5-4415-9cf8-2b88f87a9c09 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:02.370658 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a33316a-9933-43de-bf55-69f5bdbacf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:02.371299 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33316a-9933-43de-bf55-69f5bdbacf90 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:02.371617 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33316a-9933-43de-bf55-69f5bdbacf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:02.371892 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33316a-9933-43de-bf55-69f5bdbacf90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:02.372546 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a33316a-9933-43de-bf55-69f5bdbacf90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:02.374443 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a33316a-9933-43de-bf55-69f5bdbacf90 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:02.404495 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 868/1726] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:11:02 2022] POST /identity/v3/projects => generated 389 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:11:02.408874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df94644a-3c9f-4a95-85b8-0f5c1d622670 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:02.419823 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df94644a-3c9f-4a95-85b8-0f5c1d622670 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:02.476777 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7fc6f-bdb4-4ac4-9da9-e13282b23a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:02.476777 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7fc6f-bdb4-4ac4-9da9-e13282b23a10 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:02.476777 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7fc6f-bdb4-4ac4-9da9-e13282b23a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:02.476777 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7fc6f-bdb4-4ac4-9da9-e13282b23a10 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:02.478730 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f7fc6f-bdb4-4ac4-9da9-e13282b23a10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:02.480582 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f7fc6f-bdb4-4ac4-9da9-e13282b23a10 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:02.504195 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 859/1727] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:11:02 2022] POST /identity/v3/users => generated 301 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:02.508913 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a33316a-9933-43de-bf55-69f5bdbacf90 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:02.519035 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a33316a-9933-43de-bf55-69f5bdbacf90 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:02.523656 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ddfcf-b434-4381-8fd7-99e552c6c10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:02.524279 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ddfcf-b434-4381-8fd7-99e552c6c10f admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:02.524556 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ddfcf-b434-4381-8fd7-99e552c6c10f admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:02.524825 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ddfcf-b434-4381-8fd7-99e552c6c10f admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:02.525457 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ddfcf-b434-4381-8fd7-99e552c6c10f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:02.527187 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ddfcf-b434-4381-8fd7-99e552c6c10f admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:02.537537 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 869/1728] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:11:02.541850 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7fc6f-bdb4-4ac4-9da9-e13282b23a10 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:02.552912 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7fc6f-bdb4-4ac4-9da9-e13282b23a10 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:02.558259 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9976c10-c979-44ea-9c89-8b1ac2312169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:02.558909 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9976c10-c979-44ea-9c89-8b1ac2312169 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:02.559230 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9976c10-c979-44ea-9c89-8b1ac2312169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:02.559504 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9976c10-c979-44ea-9c89-8b1ac2312169 admin admin] PATH_INFO: `/v3/projects/02ed6a7deef34f7fb66f4a399e9f493a/users/966f4b1d7089420483a0059331675c9e/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:02.574733 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9976c10-c979-44ea-9c89-8b1ac2312169 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02ed6a7deef34f7fb66f4a399e9f493a, user_id=966f4b1d7089420483a0059331675c9e, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:02.577434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9976c10-c979-44ea-9c89-8b1ac2312169 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:02.590213 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 860/1729] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:02 2022] PUT /identity/v3/projects/02ed6a7deef34f7fb66f4a399e9f493a/users/966f4b1d7089420483a0059331675c9e/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:02.594135 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ddfcf-b434-4381-8fd7-99e552c6c10f admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:02.604086 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ddfcf-b434-4381-8fd7-99e552c6c10f admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:02.659128 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8c737-c5d7-40b1-af0f-37c920a09841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:02.659741 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8c737-c5d7-40b1-af0f-37c920a09841 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:02.659998 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8c737-c5d7-40b1-af0f-37c920a09841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:02.660272 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8c737-c5d7-40b1-af0f-37c920a09841 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:02.660907 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8c737-c5d7-40b1-af0f-37c920a09841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:02.662499 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8c737-c5d7-40b1-af0f-37c920a09841 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:02.672151 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 870/1730] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:02.676315 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9976c10-c979-44ea-9c89-8b1ac2312169 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:02.687202 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9976c10-c979-44ea-9c89-8b1ac2312169 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:02.691805 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c96686-7241-41a2-bbd9-c43b5df5aa7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:02.692446 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c96686-7241-41a2-bbd9-c43b5df5aa7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:02.692728 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c96686-7241-41a2-bbd9-c43b5df5aa7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:02.693002 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c96686-7241-41a2-bbd9-c43b5df5aa7b admin admin] PATH_INFO: `/v3/projects/02ed6a7deef34f7fb66f4a399e9f493a/users/966f4b1d7089420483a0059331675c9e/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:02.695561 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c96686-7241-41a2-bbd9-c43b5df5aa7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=02ed6a7deef34f7fb66f4a399e9f493a, user_id=966f4b1d7089420483a0059331675c9e, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:02.698173 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c96686-7241-41a2-bbd9-c43b5df5aa7b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:02.708188 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 861/1731] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:02 2022] PUT /identity/v3/projects/02ed6a7deef34f7fb66f4a399e9f493a/users/966f4b1d7089420483a0059331675c9e/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:11:02.719349 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbcfc9f-c9cd-44b7-989f-06a7d51660ea None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:02.719639 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbcfc9f-c9cd-44b7-989f-06a7d51660ea None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:02.719916 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbcfc9f-c9cd-44b7-989f-06a7d51660ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:02.748221 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-8dbcfc9f-c9cd-44b7-989f-06a7d51660ea None None] MFA Rules not processed for user `966f4b1d7089420483a0059331675c9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:02.749432 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-8dbcfc9f-c9cd-44b7-989f-06a7d51660ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:02.753559 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:02.819836 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 871/1732] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:02 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:02.830488 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c96686-7241-41a2-bbd9-c43b5df5aa7b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:02.832204 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-86c96686-7241-41a2-bbd9-c43b5df5aa7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:02.877736 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:02.890365 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c96686-7241-41a2-bbd9-c43b5df5aa7b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:02.893988 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0f928-bd75-4f49-8977-c70f6ed44fe7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:02.894584 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0f928-bd75-4f49-8977-c70f6ed44fe7 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:02.894864 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0f928-bd75-4f49-8977-c70f6ed44fe7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:02.895162 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0f928-bd75-4f49-8977-c70f6ed44fe7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:02.903955 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0f928-bd75-4f49-8977-c70f6ed44fe7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:02.905389 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0f928-bd75-4f49-8977-c70f6ed44fe7 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:02.915422 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 862/1733] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:02 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:11:02.965349 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbcfc9f-c9cd-44b7-989f-06a7d51660ea None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:02.966727 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-8dbcfc9f-c9cd-44b7-989f-06a7d51660ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:02.979210 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbcfc9f-c9cd-44b7-989f-06a7d51660ea None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:02.982846 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ae888-ced8-420a-9a85-40a3f7551ff1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:02.983485 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ae888-ced8-420a-9a85-40a3f7551ff1 service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:02.983767 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ae888-ced8-420a-9a85-40a3f7551ff1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:02.984040 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ae888-ced8-420a-9a85-40a3f7551ff1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:03.043454 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ae888-ced8-420a-9a85-40a3f7551ff1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:03.045049 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ae888-ced8-420a-9a85-40a3f7551ff1 service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:03.057532 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 872/1734] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:02 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:11:05.809646 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0f928-bd75-4f49-8977-c70f6ed44fe7 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:05.811068 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-47e0f928-bd75-4f49-8977-c70f6ed44fe7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:05.874840 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0f928-bd75-4f49-8977-c70f6ed44fe7 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:05.879386 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced565b-63b1-431f-b145-beb187684195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:05.880007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced565b-63b1-431f-b145-beb187684195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:05.880295 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced565b-63b1-431f-b145-beb187684195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:05.880566 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced565b-63b1-431f-b145-beb187684195 admin admin] PATH_INFO: `/v3/users/02b4ab4d1b1546a88110fe75e7b3bc45` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:05.881674 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced565b-63b1-431f-b145-beb187684195 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02b4ab4d1b1546a88110fe75e7b3bc45)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:05.883682 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced565b-63b1-431f-b145-beb187684195 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:05.931611 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-bced565b-63b1-431f-b145-beb187684195 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02b4ab4d1b1546a88110fe75e7b3bc45'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:05.938363 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-bced565b-63b1-431f-b145-beb187684195 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02b4ab4d1b1546a88110fe75e7b3bc45'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:05.939026 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-bced565b-63b1-431f-b145-beb187684195 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02b4ab4d1b1546a88110fe75e7b3bc45'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:05.946302 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-bced565b-63b1-431f-b145-beb187684195 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02b4ab4d1b1546a88110fe75e7b3bc45'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:05.957738 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-bced565b-63b1-431f-b145-beb187684195 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02b4ab4d1b1546a88110fe75e7b3bc45'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:05.963543 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 863/1735] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:11:05 2022] DELETE /identity/v3/users/02b4ab4d1b1546a88110fe75e7b3bc45 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:05.968150 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747edede-1b14-435c-9589-fe541d8a64da None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:05.968616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747edede-1b14-435c-9589-fe541d8a64da None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:05.968707 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747edede-1b14-435c-9589-fe541d8a64da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:05.989511 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-747edede-1b14-435c-9589-fe541d8a64da None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:05.990899 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-747edede-1b14-435c-9589-fe541d8a64da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:06.050919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 873/1736] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:06.061706 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced565b-63b1-431f-b145-beb187684195 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:06.063151 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-bced565b-63b1-431f-b145-beb187684195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:06.120764 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced565b-63b1-431f-b145-beb187684195 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:06.124606 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedefca7-09e3-45b7-a180-3dbcf85d189d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:06.125212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedefca7-09e3-45b7-a180-3dbcf85d189d service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:06.125507 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedefca7-09e3-45b7-a180-3dbcf85d189d service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:06.125774 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedefca7-09e3-45b7-a180-3dbcf85d189d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:06.135816 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedefca7-09e3-45b7-a180-3dbcf85d189d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:06.137325 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedefca7-09e3-45b7-a180-3dbcf85d189d service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:06.146855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 864/1737] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:11:06.548471 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747edede-1b14-435c-9589-fe541d8a64da None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:06.550011 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-747edede-1b14-435c-9589-fe541d8a64da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:06.570881 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747edede-1b14-435c-9589-fe541d8a64da None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:06.578667 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ee0f0b-e552-4d88-8bb1-81272408380a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:06.579350 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee0f0b-e552-4d88-8bb1-81272408380a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:06.579753 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee0f0b-e552-4d88-8bb1-81272408380a admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:06.579893 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee0f0b-e552-4d88-8bb1-81272408380a admin admin] PATH_INFO: `/v3/projects/a71a5ac3063d4d3d910e73a24f4d923f` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:06.581261 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ee0f0b-e552-4d88-8bb1-81272408380a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a71a5ac3063d4d3d910e73a24f4d923f)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:06.583362 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ee0f0b-e552-4d88-8bb1-81272408380a admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:06.606933 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedefca7-09e3-45b7-a180-3dbcf85d189d service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:06.609018 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-eedefca7-09e3-45b7-a180-3dbcf85d189d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:06.627026 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedefca7-09e3-45b7-a180-3dbcf85d189d service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:06.635270 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:06.635767 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03039a86-c29a-476a-a156-bb0a199e9ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:06.637929 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03039a86-c29a-476a-a156-bb0a199e9ed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:06.638735 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03039a86-c29a-476a-a156-bb0a199e9ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:06.639407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03039a86-c29a-476a-a156-bb0a199e9ed5 admin admin] PATH_INFO: `/v3/users/2c3e132b2b2e47599083c951119cac5f` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:06.640894 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03039a86-c29a-476a-a156-bb0a199e9ed5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c3e132b2b2e47599083c951119cac5f)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:06.643004 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03039a86-c29a-476a-a156-bb0a199e9ed5 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:06.645047 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:06.654085 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c2ee0f0b-e552-4d88-8bb1-81272408380a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a71a5ac3063d4d3d910e73a24f4d923f', 'request_id': 'req-c2ee0f0b-e552-4d88-8bb1-81272408380a'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:06.660723 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c2ee0f0b-e552-4d88-8bb1-81272408380a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a71a5ac3063d4d3d910e73a24f4d923f', 'request_id': 'req-c2ee0f0b-e552-4d88-8bb1-81272408380a'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:06.675892 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c2ee0f0b-e552-4d88-8bb1-81272408380a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a71a5ac3063d4d3d910e73a24f4d923f', 'request_id': 'req-c2ee0f0b-e552-4d88-8bb1-81272408380a'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:06.683939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c2ee0f0b-e552-4d88-8bb1-81272408380a admin admin] The token cache is being invalidate because project a71a5ac3063d4d3d910e73a24f4d923f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:11:06.684089 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c2ee0f0b-e552-4d88-8bb1-81272408380a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:06.686141 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c2ee0f0b-e552-4d88-8bb1-81272408380a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:06.687941 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 874/1738] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:11:06 2022] DELETE /identity/v3/projects/a71a5ac3063d4d3d910e73a24f4d923f => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:06.693967 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ed783-3fc8-4748-a095-ff7b947fe099 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:06.694306 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ed783-3fc8-4748-a095-ff7b947fe099 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:06.695454 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ed783-3fc8-4748-a095-ff7b947fe099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:06.705565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-03039a86-c29a-476a-a156-bb0a199e9ed5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c3e132b2b2e47599083c951119cac5f'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:06.714220 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-03039a86-c29a-476a-a156-bb0a199e9ed5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c3e132b2b2e47599083c951119cac5f'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:06.715110 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-03039a86-c29a-476a-a156-bb0a199e9ed5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c3e132b2b2e47599083c951119cac5f'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:06.724296 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-03039a86-c29a-476a-a156-bb0a199e9ed5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c3e132b2b2e47599083c951119cac5f'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:06.726996 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-106ed783-3fc8-4748-a095-ff7b947fe099 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:06.728280 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-106ed783-3fc8-4748-a095-ff7b947fe099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:06.738336 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-03039a86-c29a-476a-a156-bb0a199e9ed5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c3e132b2b2e47599083c951119cac5f'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:06.744286 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 865/1739] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:11:06 2022] DELETE /identity/v3/users/2c3e132b2b2e47599083c951119cac5f => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:11:06.748979 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06186926-e7a6-4d37-a1e2-dbba878eead4 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:06.749261 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06186926-e7a6-4d37-a1e2-dbba878eead4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:06.749530 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06186926-e7a6-4d37-a1e2-dbba878eead4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:06.770927 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-06186926-e7a6-4d37-a1e2-dbba878eead4 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:06.772166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-06186926-e7a6-4d37-a1e2-dbba878eead4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:06.803251 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 875/1740] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:06.805365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f690686-dd37-496c-a3db-a7e820fa414b None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:06.805661 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f690686-dd37-496c-a3db-a7e820fa414b None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:06.805937 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f690686-dd37-496c-a3db-a7e820fa414b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:06.827412 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-3f690686-dd37-496c-a3db-a7e820fa414b None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:06.828779 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-3f690686-dd37-496c-a3db-a7e820fa414b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:06.847369 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 866/1741] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:11:06.865532 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06186926-e7a6-4d37-a1e2-dbba878eead4 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:06.867584 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-06186926-e7a6-4d37-a1e2-dbba878eead4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:06.897331 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 876/1742] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:11:06.902135 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f690686-dd37-496c-a3db-a7e820fa414b None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:06.911203 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f690686-dd37-496c-a3db-a7e820fa414b None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:06.917204 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560e381-040b-4e52-ab47-869baf536239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:06.917802 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560e381-040b-4e52-ab47-869baf536239 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:06.918078 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560e381-040b-4e52-ab47-869baf536239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:06.918346 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560e381-040b-4e52-ab47-869baf536239 admin admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:06.919001 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5560e381-040b-4e52-ab47-869baf536239 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:11:06.919365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5560e381-040b-4e52-ab47-869baf536239 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:06.921093 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5560e381-040b-4e52-ab47-869baf536239 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:06.945494 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 877/1743] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:11:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 13:11:06.945776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06186926-e7a6-4d37-a1e2-dbba878eead4 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:06.949191 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19198f9b-1523-423d-bbb3-5a6294c4394f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:06.949587 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560e381-040b-4e52-ab47-869baf536239 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:06.949797 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19198f9b-1523-423d-bbb3-5a6294c4394f service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:06.950079 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19198f9b-1523-423d-bbb3-5a6294c4394f service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:06.950359 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19198f9b-1523-423d-bbb3-5a6294c4394f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:06.959595 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19198f9b-1523-423d-bbb3-5a6294c4394f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:06.959753 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560e381-040b-4e52-ab47-869baf536239 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:06.961588 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19198f9b-1523-423d-bbb3-5a6294c4394f service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:06.964725 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e404b7c-cf65-43bb-9de6-a7cc6e0ea7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:06.965312 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e404b7c-cf65-43bb-9de6-a7cc6e0ea7cc admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:06.965591 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e404b7c-cf65-43bb-9de6-a7cc6e0ea7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:06.965867 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e404b7c-cf65-43bb-9de6-a7cc6e0ea7cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:06.966480 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e404b7c-cf65-43bb-9de6-a7cc6e0ea7cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:06.968404 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e404b7c-cf65-43bb-9de6-a7cc6e0ea7cc admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:06.971122 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 867/1744] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 13 13:11:06.997792 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 878/1745] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:11:06 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:11:07.001916 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19198f9b-1523-423d-bbb3-5a6294c4394f service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:07.012432 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19198f9b-1523-423d-bbb3-5a6294c4394f service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:07.034457 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:07.078545 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9b2cf-fe6c-4725-84b2-e8f47dbd375c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:07.079191 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9b2cf-fe6c-4725-84b2-e8f47dbd375c admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:07.079472 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9b2cf-fe6c-4725-84b2-e8f47dbd375c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:07.079748 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9b2cf-fe6c-4725-84b2-e8f47dbd375c admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:07.080397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da9b2cf-fe6c-4725-84b2-e8f47dbd375c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:07.082310 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da9b2cf-fe6c-4725-84b2-e8f47dbd375c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:07.105438 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 868/1746] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:11:07 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:11:07.109610 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e404b7c-cf65-43bb-9de6-a7cc6e0ea7cc admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:07.124022 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e404b7c-cf65-43bb-9de6-a7cc6e0ea7cc admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:07.132850 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518c2fe-7372-4ede-b406-55eb415633b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:07.133887 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518c2fe-7372-4ede-b406-55eb415633b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:07.134416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518c2fe-7372-4ede-b406-55eb415633b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:07.134903 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518c2fe-7372-4ede-b406-55eb415633b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:07.136122 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518c2fe-7372-4ede-b406-55eb415633b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:07.139181 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518c2fe-7372-4ede-b406-55eb415633b8 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:07.152812 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 879/1747] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:07.156914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9b2cf-fe6c-4725-84b2-e8f47dbd375c admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:07.167816 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9b2cf-fe6c-4725-84b2-e8f47dbd375c admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:07.172860 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2825c6-d096-4676-92d4-35e51576b926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:07.173427 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2825c6-d096-4676-92d4-35e51576b926 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:07.173688 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2825c6-d096-4676-92d4-35e51576b926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:07.173954 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2825c6-d096-4676-92d4-35e51576b926 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:07.174616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2825c6-d096-4676-92d4-35e51576b926 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:07.176549 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2825c6-d096-4676-92d4-35e51576b926 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:07.188143 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 869/1748] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:07.192285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518c2fe-7372-4ede-b406-55eb415633b8 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:07.204278 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518c2fe-7372-4ede-b406-55eb415633b8 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:07.209009 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fcb2a6-a585-48bf-a238-5e13c99a7eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:07.209886 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fcb2a6-a585-48bf-a238-5e13c99a7eba admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:07.210213 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fcb2a6-a585-48bf-a238-5e13c99a7eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:07.210495 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fcb2a6-a585-48bf-a238-5e13c99a7eba admin admin] PATH_INFO: `/v3/projects/f4e890d2225c4489bae4f0887e6ade6d/users/f808b5b67e554673bffd80a67ee97afb/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:07.226439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fcb2a6-a585-48bf-a238-5e13c99a7eba admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4e890d2225c4489bae4f0887e6ade6d, user_id=f808b5b67e554673bffd80a67ee97afb, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:07.229296 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fcb2a6-a585-48bf-a238-5e13c99a7eba admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:07.231927 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:07.254725 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 880/1749] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:07 2022] PUT /identity/v3/projects/f4e890d2225c4489bae4f0887e6ade6d/users/f808b5b67e554673bffd80a67ee97afb/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:07.256500 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d020f-e2c1-4cf0-b4b9-36e1314aff91 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:07.256912 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d020f-e2c1-4cf0-b4b9-36e1314aff91 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:07.257774 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d020f-e2c1-4cf0-b4b9-36e1314aff91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:07.286855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fcb2a6-a585-48bf-a238-5e13c99a7eba admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:07.291956 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-d6fcb2a6-a585-48bf-a238-5e13c99a7eba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:07.294540 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-713d020f-e2c1-4cf0-b4b9-36e1314aff91 None None] MFA Rules not processed for user `f808b5b67e554673bffd80a67ee97afb`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:07.298052 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-713d020f-e2c1-4cf0-b4b9-36e1314aff91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:07.374773 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 870/1750] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:07.379167 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d020f-e2c1-4cf0-b4b9-36e1314aff91 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:07.386679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fcb2a6-a585-48bf-a238-5e13c99a7eba admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:07.389252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d020f-e2c1-4cf0-b4b9-36e1314aff91 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:07.391386 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c822b68-8da2-4309-bf0d-61a041eaea4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:07.392105 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c822b68-8da2-4309-bf0d-61a041eaea4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:07.392472 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c822b68-8da2-4309-bf0d-61a041eaea4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:07.392810 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c822b68-8da2-4309-bf0d-61a041eaea4e admin admin] PATH_INFO: `/v3/projects/f2509c4b1f634b1a96668b9c697d9bdf` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:07.393788 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3aef6c-4282-41c3-86d8-7369f56b01e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:07.394057 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c822b68-8da2-4309-bf0d-61a041eaea4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2509c4b1f634b1a96668b9c697d9bdf)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:07.394372 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3aef6c-4282-41c3-86d8-7369f56b01e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:07.394654 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3aef6c-4282-41c3-86d8-7369f56b01e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:07.394932 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3aef6c-4282-41c3-86d8-7369f56b01e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:07.395598 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3aef6c-4282-41c3-86d8-7369f56b01e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:07.396435 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c822b68-8da2-4309-bf0d-61a041eaea4e admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:07.397537 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3aef6c-4282-41c3-86d8-7369f56b01e8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:07.424738 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 871/1751] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:11:07 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:11:07.429589 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3aef6c-4282-41c3-86d8-7369f56b01e8 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:07.444434 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-6c822b68-8da2-4309-bf0d-61a041eaea4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2509c4b1f634b1a96668b9c697d9bdf', 'request_id': 'req-6c822b68-8da2-4309-bf0d-61a041eaea4e'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:07.450234 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-6c822b68-8da2-4309-bf0d-61a041eaea4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2509c4b1f634b1a96668b9c697d9bdf', 'request_id': 'req-6c822b68-8da2-4309-bf0d-61a041eaea4e'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:07.456843 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-6c822b68-8da2-4309-bf0d-61a041eaea4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2509c4b1f634b1a96668b9c697d9bdf', 'request_id': 'req-6c822b68-8da2-4309-bf0d-61a041eaea4e'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:07.478638 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-6c822b68-8da2-4309-bf0d-61a041eaea4e admin admin] The token cache is being invalidate because project f2509c4b1f634b1a96668b9c697d9bdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:11:07.479016 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-6c822b68-8da2-4309-bf0d-61a041eaea4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:07.479768 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-6c822b68-8da2-4309-bf0d-61a041eaea4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:07.481595 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 881/1752] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:11:07 2022] DELETE /identity/v3/projects/f2509c4b1f634b1a96668b9c697d9bdf => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:11:07.485429 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c822b68-8da2-4309-bf0d-61a041eaea4e admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:07.487027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-6c822b68-8da2-4309-bf0d-61a041eaea4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:07.496634 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3aef6c-4282-41c3-86d8-7369f56b01e8 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:07.510268 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:07.565393 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9cbad-294d-4c2a-b66b-e5e75682c3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:07.565844 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c822b68-8da2-4309-bf0d-61a041eaea4e admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:07.566342 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9cbad-294d-4c2a-b66b-e5e75682c3aa admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:07.566607 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9cbad-294d-4c2a-b66b-e5e75682c3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:07.566886 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9cbad-294d-4c2a-b66b-e5e75682c3aa admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:07.568296 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9cbad-294d-4c2a-b66b-e5e75682c3aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:07.570934 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9cbad-294d-4c2a-b66b-e5e75682c3aa admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:07.572906 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c280fad3-84d5-456b-87a6-84b2f2b6222c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:07.572906 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280fad3-84d5-456b-87a6-84b2f2b6222c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:07.572906 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280fad3-84d5-456b-87a6-84b2f2b6222c admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:07.572906 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280fad3-84d5-456b-87a6-84b2f2b6222c admin admin] PATH_INFO: `/v3/users/966f4b1d7089420483a0059331675c9e` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:07.574151 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c280fad3-84d5-456b-87a6-84b2f2b6222c admin admin] RBAC: Authorizing `identity:delete_user(user_id=966f4b1d7089420483a0059331675c9e)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:07.576139 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c280fad3-84d5-456b-87a6-84b2f2b6222c admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:07.599300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 872/1753] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:11:07 2022] POST /identity/v3/users => generated 294 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:11:07.604010 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9cbad-294d-4c2a-b66b-e5e75682c3aa admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:07.606045 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5ee9cbad-294d-4c2a-b66b-e5e75682c3aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:07.612414 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:INSERT=2,keystone:SELECT=87,keystone:DELETE=11 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:07.621448 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9cbad-294d-4c2a-b66b-e5e75682c3aa admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:07.626646 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48939d-25ee-4dee-a083-1eb0b90d50fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:07.627296 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48939d-25ee-4dee-a083-1eb0b90d50fe admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:07.627577 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48939d-25ee-4dee-a083-1eb0b90d50fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:07.627846 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48939d-25ee-4dee-a083-1eb0b90d50fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:07.628485 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e48939d-25ee-4dee-a083-1eb0b90d50fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:07.630062 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e48939d-25ee-4dee-a083-1eb0b90d50fe admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:07.637103 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c280fad3-84d5-456b-87a6-84b2f2b6222c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '966f4b1d7089420483a0059331675c9e'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:07.643385 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 873/1754] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:11:07.648094 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48939d-25ee-4dee-a083-1eb0b90d50fe admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:07.649267 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c280fad3-84d5-456b-87a6-84b2f2b6222c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '966f4b1d7089420483a0059331675c9e'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:07.651909 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c280fad3-84d5-456b-87a6-84b2f2b6222c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '966f4b1d7089420483a0059331675c9e'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:07.652551 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c280fad3-84d5-456b-87a6-84b2f2b6222c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '966f4b1d7089420483a0059331675c9e'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:07.659554 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48939d-25ee-4dee-a083-1eb0b90d50fe admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:07.660972 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c280fad3-84d5-456b-87a6-84b2f2b6222c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '966f4b1d7089420483a0059331675c9e'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:07.664968 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54824bde-e59e-43ce-a7dc-2e35b6eeaccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:07.665561 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54824bde-e59e-43ce-a7dc-2e35b6eeaccb admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:07.665858 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54824bde-e59e-43ce-a7dc-2e35b6eeaccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:07.666119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54824bde-e59e-43ce-a7dc-2e35b6eeaccb admin admin] PATH_INFO: `/v3/projects/1dd734ee08c6447da7b850631a4a8e6a/users/5fdb7983e56148528acbd6e24ec3e927/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:07.683241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54824bde-e59e-43ce-a7dc-2e35b6eeaccb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dd734ee08c6447da7b850631a4a8e6a, user_id=5fdb7983e56148528acbd6e24ec3e927, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:07.685138 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 882/1755] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:11:07 2022] DELETE /identity/v3/users/966f4b1d7089420483a0059331675c9e => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:11:07.686780 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54824bde-e59e-43ce-a7dc-2e35b6eeaccb admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:07.702917 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 874/1756] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:07 2022] PUT /identity/v3/projects/1dd734ee08c6447da7b850631a4a8e6a/users/5fdb7983e56148528acbd6e24ec3e927/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:11:07.707087 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c280fad3-84d5-456b-87a6-84b2f2b6222c admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:07.708626 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-c280fad3-84d5-456b-87a6-84b2f2b6222c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:07.792550 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c280fad3-84d5-456b-87a6-84b2f2b6222c admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:07.875636 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d278d6-f0a7-4f65-9583-f9a798196bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:07.876299 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d278d6-f0a7-4f65-9583-f9a798196bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:07.876525 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d278d6-f0a7-4f65-9583-f9a798196bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:07.876794 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d278d6-f0a7-4f65-9583-f9a798196bb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:07.877481 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d278d6-f0a7-4f65-9583-f9a798196bb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:07.879082 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d278d6-f0a7-4f65-9583-f9a798196bb4 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:07.891360 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 883/1757] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:07.895453 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54824bde-e59e-43ce-a7dc-2e35b6eeaccb admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:07.905808 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54824bde-e59e-43ce-a7dc-2e35b6eeaccb admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:07.912007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f53e5-688e-462a-9660-26298a1010e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:07.912860 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f53e5-688e-462a-9660-26298a1010e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:07.913784 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f53e5-688e-462a-9660-26298a1010e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:07.913784 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f53e5-688e-462a-9660-26298a1010e2 admin admin] PATH_INFO: `/v3/projects/1dd734ee08c6447da7b850631a4a8e6a/users/5fdb7983e56148528acbd6e24ec3e927/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:07.916188 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f53e5-688e-462a-9660-26298a1010e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dd734ee08c6447da7b850631a4a8e6a, user_id=5fdb7983e56148528acbd6e24ec3e927, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:07.919225 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f53e5-688e-462a-9660-26298a1010e2 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:07.938021 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 875/1758] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:07 2022] PUT /identity/v3/projects/1dd734ee08c6447da7b850631a4a8e6a/users/5fdb7983e56148528acbd6e24ec3e927/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:07.949854 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87e801-d472-4f87-b753-b5d1bafd2f98 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:07.950143 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87e801-d472-4f87-b753-b5d1bafd2f98 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:07.950419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87e801-d472-4f87-b753-b5d1bafd2f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:07.980527 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-3b87e801-d472-4f87-b753-b5d1bafd2f98 None None] MFA Rules not processed for user `5fdb7983e56148528acbd6e24ec3e927`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:07.981739 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-3b87e801-d472-4f87-b753-b5d1bafd2f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:08.029626 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:08.076607 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 884/1759] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:07 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:08.088917 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f53e5-688e-462a-9660-26298a1010e2 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:08.091950 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3d3f53e5-688e-462a-9660-26298a1010e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:08.166755 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f53e5-688e-462a-9660-26298a1010e2 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:08.170641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3817e-c441-4a14-8299-4d87dc4c247d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:08.171298 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3817e-c441-4a14-8299-4d87dc4c247d service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:08.171610 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3817e-c441-4a14-8299-4d87dc4c247d service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:08.171882 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3817e-c441-4a14-8299-4d87dc4c247d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:08.173627 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-25f3817e-c441-4a14-8299-4d87dc4c247d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:08.236340 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f3817e-c441-4a14-8299-4d87dc4c247d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:08.238153 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f3817e-c441-4a14-8299-4d87dc4c247d service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:08.248969 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 876/1760] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:11:08.486824 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b87e801-d472-4f87-b753-b5d1bafd2f98 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:08.488285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-3b87e801-d472-4f87-b753-b5d1bafd2f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:08.644325 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b87e801-d472-4f87-b753-b5d1bafd2f98 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:08.679357 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a473bc8-9b40-4ea1-bb96-5087dd6673b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:08.679357 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a473bc8-9b40-4ea1-bb96-5087dd6673b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:08.679357 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a473bc8-9b40-4ea1-bb96-5087dd6673b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:08.679357 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a473bc8-9b40-4ea1-bb96-5087dd6673b1 admin admin] PATH_INFO: `/v3/projects/02ed6a7deef34f7fb66f4a399e9f493a` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:08.683009 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a473bc8-9b40-4ea1-bb96-5087dd6673b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02ed6a7deef34f7fb66f4a399e9f493a)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:08.685443 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a473bc8-9b40-4ea1-bb96-5087dd6673b1 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:08.762750 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-2a473bc8-9b40-4ea1-bb96-5087dd6673b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02ed6a7deef34f7fb66f4a399e9f493a', 'request_id': 'req-2a473bc8-9b40-4ea1-bb96-5087dd6673b1'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:08.770016 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-2a473bc8-9b40-4ea1-bb96-5087dd6673b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02ed6a7deef34f7fb66f4a399e9f493a', 'request_id': 'req-2a473bc8-9b40-4ea1-bb96-5087dd6673b1'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:08.777819 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-2a473bc8-9b40-4ea1-bb96-5087dd6673b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02ed6a7deef34f7fb66f4a399e9f493a', 'request_id': 'req-2a473bc8-9b40-4ea1-bb96-5087dd6673b1'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:08.795288 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-2a473bc8-9b40-4ea1-bb96-5087dd6673b1 admin admin] The token cache is being invalidate because project 02ed6a7deef34f7fb66f4a399e9f493a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:11:08.795654 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-2a473bc8-9b40-4ea1-bb96-5087dd6673b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:08.796449 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-2a473bc8-9b40-4ea1-bb96-5087dd6673b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:08.798630 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 885/1761] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:11:08 2022] DELETE /identity/v3/projects/02ed6a7deef34f7fb66f4a399e9f493a => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:08.803828 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1479eaa-17e4-4a34-9e2a-d74cde354a59 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:08.804067 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1479eaa-17e4-4a34-9e2a-d74cde354a59 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:08.804335 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1479eaa-17e4-4a34-9e2a-d74cde354a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:08.831118 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c1479eaa-17e4-4a34-9e2a-d74cde354a59 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:08.832327 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c1479eaa-17e4-4a34-9e2a-d74cde354a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:08.838402 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:08.894343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 877/1762] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:08.902436 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7aacb-a499-434d-8b9e-2a4af677c2df None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:08.902723 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7aacb-a499-434d-8b9e-2a4af677c2df None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:08.903007 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7aacb-a499-434d-8b9e-2a4af677c2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:08.938447 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-9ee7aacb-a499-434d-8b9e-2a4af677c2df None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:08.939665 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-9ee7aacb-a499-434d-8b9e-2a4af677c2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:08.948193 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 886/1763] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:08.952567 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1479eaa-17e4-4a34-9e2a-d74cde354a59 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:08.963365 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1479eaa-17e4-4a34-9e2a-d74cde354a59 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:08.969216 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53458427-51b1-4e62-899e-0e03b98eeeea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:08.970001 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53458427-51b1-4e62-899e-0e03b98eeeea admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:08.970427 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53458427-51b1-4e62-899e-0e03b98eeeea admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:08.970869 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53458427-51b1-4e62-899e-0e03b98eeeea admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:08.971800 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53458427-51b1-4e62-899e-0e03b98eeeea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:11:08.972247 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53458427-51b1-4e62-899e-0e03b98eeeea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:08.974133 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53458427-51b1-4e62-899e-0e03b98eeeea admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:08.987540 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 878/1764] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:11:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:11:08.991624 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee7aacb-a499-434d-8b9e-2a4af677c2df None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:09.002778 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee7aacb-a499-434d-8b9e-2a4af677c2df None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:09.011633 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a6c09a-7155-405e-9ac8-b2aa57e7b6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:09.013005 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6c09a-7155-405e-9ac8-b2aa57e7b6d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:09.013525 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6c09a-7155-405e-9ac8-b2aa57e7b6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:09.014067 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6c09a-7155-405e-9ac8-b2aa57e7b6d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:09.015214 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a6c09a-7155-405e-9ac8-b2aa57e7b6d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:09.017619 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a6c09a-7155-405e-9ac8-b2aa57e7b6d9 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:09.059631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 887/1765] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:11:08 2022] POST /identity/v3/projects => generated 399 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:09.064144 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53458427-51b1-4e62-899e-0e03b98eeeea admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:09.077868 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53458427-51b1-4e62-899e-0e03b98eeeea admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:09.157389 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88504c3e-cb85-4b56-bdf2-9d9d541e3f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:09.158259 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88504c3e-cb85-4b56-bdf2-9d9d541e3f20 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:09.159699 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88504c3e-cb85-4b56-bdf2-9d9d541e3f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:09.161402 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88504c3e-cb85-4b56-bdf2-9d9d541e3f20 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:09.162287 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88504c3e-cb85-4b56-bdf2-9d9d541e3f20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:09.168197 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88504c3e-cb85-4b56-bdf2-9d9d541e3f20 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:09.216521 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 879/1766] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:11:09 2022] POST /identity/v3/users => generated 300 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:09.222014 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a6c09a-7155-405e-9ac8-b2aa57e7b6d9 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:09.247657 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a6c09a-7155-405e-9ac8-b2aa57e7b6d9 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:09.256557 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12207255-7f9a-4690-be53-74667764b4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:09.257571 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12207255-7f9a-4690-be53-74667764b4c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:09.257916 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12207255-7f9a-4690-be53-74667764b4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:09.258253 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12207255-7f9a-4690-be53-74667764b4c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:09.259326 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12207255-7f9a-4690-be53-74667764b4c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:09.261834 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12207255-7f9a-4690-be53-74667764b4c5 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:09.306448 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 888/1767] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:09.311124 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88504c3e-cb85-4b56-bdf2-9d9d541e3f20 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:09.326592 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88504c3e-cb85-4b56-bdf2-9d9d541e3f20 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:09.335128 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6925e7fb-201d-4c59-8d7c-6a33fa81841d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:09.335128 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925e7fb-201d-4c59-8d7c-6a33fa81841d admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:09.335128 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925e7fb-201d-4c59-8d7c-6a33fa81841d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:09.335398 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6925e7fb-201d-4c59-8d7c-6a33fa81841d admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:09.340731 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6925e7fb-201d-4c59-8d7c-6a33fa81841d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:09.342849 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6925e7fb-201d-4c59-8d7c-6a33fa81841d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:09.359095 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 880/1768] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:09.363147 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12207255-7f9a-4690-be53-74667764b4c5 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:09.374316 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12207255-7f9a-4690-be53-74667764b4c5 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:09.381810 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e78aa7-9d30-477e-8afa-8c1083d7b3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:09.383489 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e78aa7-9d30-477e-8afa-8c1083d7b3d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:09.383813 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e78aa7-9d30-477e-8afa-8c1083d7b3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:09.384162 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e78aa7-9d30-477e-8afa-8c1083d7b3d8 admin admin] PATH_INFO: `/v3/projects/adf09d575b3844f594773d709f44ff92/users/56aeb18c2d114a42a3a8fb70c95027f6/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:09.403222 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e78aa7-9d30-477e-8afa-8c1083d7b3d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adf09d575b3844f594773d709f44ff92, user_id=56aeb18c2d114a42a3a8fb70c95027f6, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:09.405989 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e78aa7-9d30-477e-8afa-8c1083d7b3d8 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:09.415109 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:09.423182 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 889/1769] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:09 2022] PUT /identity/v3/projects/adf09d575b3844f594773d709f44ff92/users/56aeb18c2d114a42a3a8fb70c95027f6/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:09.435896 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd40f3-7201-4de1-9433-ec6d911553e1 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:09.436612 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd40f3-7201-4de1-9433-ec6d911553e1 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:09.437105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd40f3-7201-4de1-9433-ec6d911553e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:09.468230 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-febd40f3-7201-4de1-9433-ec6d911553e1 None None] MFA Rules not processed for user `56aeb18c2d114a42a3a8fb70c95027f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:09.469496 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-febd40f3-7201-4de1-9433-ec6d911553e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:09.550725 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 881/1770] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:09 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:09.555676 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e78aa7-9d30-477e-8afa-8c1083d7b3d8 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:09.644668 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e78aa7-9d30-477e-8afa-8c1083d7b3d8 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:09.645655 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf1828-8d15-4a8a-bf27-32190f9400b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:09.645655 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf1828-8d15-4a8a-bf27-32190f9400b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:09.645849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf1828-8d15-4a8a-bf27-32190f9400b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:09.645934 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf1828-8d15-4a8a-bf27-32190f9400b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:09.646724 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf1828-8d15-4a8a-bf27-32190f9400b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:09.650443 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf1828-8d15-4a8a-bf27-32190f9400b2 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:09.713276 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 890/1771] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:11:09 2022] POST /identity/v3/projects => generated 401 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:09.717338 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febd40f3-7201-4de1-9433-ec6d911553e1 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:09.736391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febd40f3-7201-4de1-9433-ec6d911553e1 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:09.747927 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:09.841427 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f3eba-54d5-43b9-adf5-a6f7dd92a218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:09.842057 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f3eba-54d5-43b9-adf5-a6f7dd92a218 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:09.842337 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f3eba-54d5-43b9-adf5-a6f7dd92a218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:09.842602 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f3eba-54d5-43b9-adf5-a6f7dd92a218 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:09.843558 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f3eba-54d5-43b9-adf5-a6f7dd92a218 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:09.852319 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f3eba-54d5-43b9-adf5-a6f7dd92a218 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:09.881168 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 882/1772] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:11:09 2022] POST /identity/v3/users => generated 307 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:09.886036 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf1828-8d15-4a8a-bf27-32190f9400b2 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:09.896959 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf1828-8d15-4a8a-bf27-32190f9400b2 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:09.902593 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e0082-1985-4afd-9da0-c2941eb4f202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:09.903345 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e0082-1985-4afd-9da0-c2941eb4f202 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:09.904902 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e0082-1985-4afd-9da0-c2941eb4f202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:09.905544 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e0082-1985-4afd-9da0-c2941eb4f202 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:09.906333 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e0082-1985-4afd-9da0-c2941eb4f202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:09.908439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e0082-1985-4afd-9da0-c2941eb4f202 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:09.928097 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 891/1773] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:09.932718 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f3eba-54d5-43b9-adf5-a6f7dd92a218 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:09.947210 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f3eba-54d5-43b9-adf5-a6f7dd92a218 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:09.954056 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54642eec-bda1-44cc-a485-0c3bdbe8e02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:09.955796 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54642eec-bda1-44cc-a485-0c3bdbe8e02d admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:09.956322 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54642eec-bda1-44cc-a485-0c3bdbe8e02d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:09.956719 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54642eec-bda1-44cc-a485-0c3bdbe8e02d admin admin] PATH_INFO: `/v3/projects/df202d6d34a14d03acc9afd3b6ad1c25/users/ca0698594c13415a98167b73e36931a9/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:09.979178 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54642eec-bda1-44cc-a485-0c3bdbe8e02d admin admin] RBAC: Authorizing `identity:create_grant(project_id=df202d6d34a14d03acc9afd3b6ad1c25, user_id=ca0698594c13415a98167b73e36931a9, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:09.982482 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54642eec-bda1-44cc-a485-0c3bdbe8e02d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:10.004394 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 883/1774] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:09 2022] PUT /identity/v3/projects/df202d6d34a14d03acc9afd3b6ad1c25/users/ca0698594c13415a98167b73e36931a9/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:10.009049 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e0082-1985-4afd-9da0-c2941eb4f202 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:10.028970 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e0082-1985-4afd-9da0-c2941eb4f202 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:10.097659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d132563-67da-4083-9f14-10bbb3a70418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:10.098276 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d132563-67da-4083-9f14-10bbb3a70418 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:10.098577 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d132563-67da-4083-9f14-10bbb3a70418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:10.098877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d132563-67da-4083-9f14-10bbb3a70418 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:10.099573 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d132563-67da-4083-9f14-10bbb3a70418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:10.101205 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d132563-67da-4083-9f14-10bbb3a70418 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:10.115327 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 892/1775] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:11:10.117991 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54642eec-bda1-44cc-a485-0c3bdbe8e02d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:10.131343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54642eec-bda1-44cc-a485-0c3bdbe8e02d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:10.136835 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629545b-5410-439a-9207-c2bd35e488eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:10.137627 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629545b-5410-439a-9207-c2bd35e488eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:10.137985 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629545b-5410-439a-9207-c2bd35e488eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:10.138323 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629545b-5410-439a-9207-c2bd35e488eb admin admin] PATH_INFO: `/v3/projects/df202d6d34a14d03acc9afd3b6ad1c25/users/ca0698594c13415a98167b73e36931a9/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:10.140677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4629545b-5410-439a-9207-c2bd35e488eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=df202d6d34a14d03acc9afd3b6ad1c25, user_id=ca0698594c13415a98167b73e36931a9, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:10.143690 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4629545b-5410-439a-9207-c2bd35e488eb admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:10.154748 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 884/1776] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:10 2022] PUT /identity/v3/projects/df202d6d34a14d03acc9afd3b6ad1c25/users/ca0698594c13415a98167b73e36931a9/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:10.165275 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee8635-a6f8-4899-a8f6-d84d278c6533 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:10.165562 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee8635-a6f8-4899-a8f6-d84d278c6533 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:10.165833 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee8635-a6f8-4899-a8f6-d84d278c6533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:10.194141 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-28ee8635-a6f8-4899-a8f6-d84d278c6533 None None] MFA Rules not processed for user `ca0698594c13415a98167b73e36931a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:10.195975 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-28ee8635-a6f8-4899-a8f6-d84d278c6533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:10.253395 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:10.264724 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 893/1777] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:10 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:10.275571 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629545b-5410-439a-9207-c2bd35e488eb admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:10.277290 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4629545b-5410-439a-9207-c2bd35e488eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:10.344302 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629545b-5410-439a-9207-c2bd35e488eb admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:10.348058 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd7e45-b287-43d1-bc84-9ae4cf989a8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:10.348656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd7e45-b287-43d1-bc84-9ae4cf989a8a service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:10.348952 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd7e45-b287-43d1-bc84-9ae4cf989a8a service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:10.349543 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd7e45-b287-43d1-bc84-9ae4cf989a8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:10.360414 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd7e45-b287-43d1-bc84-9ae4cf989a8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:10.362197 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd7e45-b287-43d1-bc84-9ae4cf989a8a service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:10.376032 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 885/1778] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:10 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:11:10.473800 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee8635-a6f8-4899-a8f6-d84d278c6533 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:10.484298 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee8635-a6f8-4899-a8f6-d84d278c6533 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:10.488316 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ecdf5-04c0-4073-b1ee-24a5f83a92fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:10.489015 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ecdf5-04c0-4073-b1ee-24a5f83a92fe service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:10.489366 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ecdf5-04c0-4073-b1ee-24a5f83a92fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:10.489740 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ecdf5-04c0-4073-b1ee-24a5f83a92fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:10.561619 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ecdf5-04c0-4073-b1ee-24a5f83a92fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:10.563015 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ecdf5-04c0-4073-b1ee-24a5f83a92fe service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:10.572217 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 894/1779] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:10 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:11:11.059839 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd7e45-b287-43d1-bc84-9ae4cf989a8a service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:11.061230 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-befd7e45-b287-43d1-bc84-9ae4cf989a8a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:11.127127 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd7e45-b287-43d1-bc84-9ae4cf989a8a service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:11.133438 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c016d00-83ed-44fd-84ec-5d31bdfd8fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:11.134063 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c016d00-83ed-44fd-84ec-5d31bdfd8fcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:11.134343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c016d00-83ed-44fd-84ec-5d31bdfd8fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:11.134611 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c016d00-83ed-44fd-84ec-5d31bdfd8fcc admin admin] PATH_INFO: `/v3/users/f808b5b67e554673bffd80a67ee97afb` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:11.135734 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c016d00-83ed-44fd-84ec-5d31bdfd8fcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f808b5b67e554673bffd80a67ee97afb)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:11.137662 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c016d00-83ed-44fd-84ec-5d31bdfd8fcc admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:11.139967 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:11.187375 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3c016d00-83ed-44fd-84ec-5d31bdfd8fcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f808b5b67e554673bffd80a67ee97afb'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:11.196246 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3c016d00-83ed-44fd-84ec-5d31bdfd8fcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f808b5b67e554673bffd80a67ee97afb'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:11.196936 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3c016d00-83ed-44fd-84ec-5d31bdfd8fcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f808b5b67e554673bffd80a67ee97afb'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:11.204548 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3c016d00-83ed-44fd-84ec-5d31bdfd8fcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f808b5b67e554673bffd80a67ee97afb'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:11.216943 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3c016d00-83ed-44fd-84ec-5d31bdfd8fcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f808b5b67e554673bffd80a67ee97afb'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:11.222731 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 886/1780] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:11:11 2022] DELETE /identity/v3/users/f808b5b67e554673bffd80a67ee97afb => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:11.227243 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b18df-3eed-4823-843d-0e4f191222af None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:11.227709 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b18df-3eed-4823-843d-0e4f191222af None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:11.227826 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b18df-3eed-4823-843d-0e4f191222af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:11.246642 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-6f9b18df-3eed-4823-843d-0e4f191222af None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:11.247869 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-6f9b18df-3eed-4823-843d-0e4f191222af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:11.307263 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 895/1781] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:11.317108 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c016d00-83ed-44fd-84ec-5d31bdfd8fcc admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:11.318496 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3c016d00-83ed-44fd-84ec-5d31bdfd8fcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:11.379779 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c016d00-83ed-44fd-84ec-5d31bdfd8fcc admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:11.384065 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff58e8-026b-4579-ad9d-23dc8bf5d78b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:11.384679 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff58e8-026b-4579-ad9d-23dc8bf5d78b service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:11.384959 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff58e8-026b-4579-ad9d-23dc8bf5d78b service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:11.385233 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff58e8-026b-4579-ad9d-23dc8bf5d78b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:11.394547 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ff58e8-026b-4579-ad9d-23dc8bf5d78b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:11.396005 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ff58e8-026b-4579-ad9d-23dc8bf5d78b service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:11.405940 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 887/1782] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:11:11.687161 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b18df-3eed-4823-843d-0e4f191222af None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:11.688592 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-6f9b18df-3eed-4823-843d-0e4f191222af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:11.704606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b18df-3eed-4823-843d-0e4f191222af None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:11.709406 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73b2d8e-e784-4d66-b075-2e0dd861957a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:11.710051 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b2d8e-e784-4d66-b075-2e0dd861957a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:11.710329 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b2d8e-e784-4d66-b075-2e0dd861957a admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:11.710605 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b2d8e-e784-4d66-b075-2e0dd861957a admin admin] PATH_INFO: `/v3/projects/f4e890d2225c4489bae4f0887e6ade6d` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:11.711772 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73b2d8e-e784-4d66-b075-2e0dd861957a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4e890d2225c4489bae4f0887e6ade6d)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:11.713714 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73b2d8e-e784-4d66-b075-2e0dd861957a admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:11.770237 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c73b2d8e-e784-4d66-b075-2e0dd861957a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4e890d2225c4489bae4f0887e6ade6d', 'request_id': 'req-c73b2d8e-e784-4d66-b075-2e0dd861957a'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:11.775662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c73b2d8e-e784-4d66-b075-2e0dd861957a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4e890d2225c4489bae4f0887e6ade6d', 'request_id': 'req-c73b2d8e-e784-4d66-b075-2e0dd861957a'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:11.787465 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c73b2d8e-e784-4d66-b075-2e0dd861957a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4e890d2225c4489bae4f0887e6ade6d', 'request_id': 'req-c73b2d8e-e784-4d66-b075-2e0dd861957a'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:11.805736 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c73b2d8e-e784-4d66-b075-2e0dd861957a admin admin] The token cache is being invalidate because project f4e890d2225c4489bae4f0887e6ade6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:11:11.807188 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c73b2d8e-e784-4d66-b075-2e0dd861957a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:11.808342 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c73b2d8e-e784-4d66-b075-2e0dd861957a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:11.810269 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 896/1783] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:11:11 2022] DELETE /identity/v3/projects/f4e890d2225c4489bae4f0887e6ade6d => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:11.815497 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff58e8-026b-4579-ad9d-23dc8bf5d78b service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:11.833620 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff58e8-026b-4579-ad9d-23dc8bf5d78b service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:11.897946 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ca781-395a-416e-bfc7-db63e467ff36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:11.898686 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ca781-395a-416e-bfc7-db63e467ff36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:11.899092 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ca781-395a-416e-bfc7-db63e467ff36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:11.899572 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ca781-395a-416e-bfc7-db63e467ff36 admin admin] PATH_INFO: `/v3/users/5fdb7983e56148528acbd6e24ec3e927` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:11.901041 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ca781-395a-416e-bfc7-db63e467ff36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fdb7983e56148528acbd6e24ec3e927)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:11.905604 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ca781-395a-416e-bfc7-db63e467ff36 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:11.946750 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:11.981476 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-237ca781-395a-416e-bfc7-db63e467ff36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fdb7983e56148528acbd6e24ec3e927'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:11.992680 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-237ca781-395a-416e-bfc7-db63e467ff36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fdb7983e56148528acbd6e24ec3e927'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:11.993453 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-237ca781-395a-416e-bfc7-db63e467ff36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fdb7983e56148528acbd6e24ec3e927'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:12.013726 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-237ca781-395a-416e-bfc7-db63e467ff36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fdb7983e56148528acbd6e24ec3e927'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:12.049166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-237ca781-395a-416e-bfc7-db63e467ff36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fdb7983e56148528acbd6e24ec3e927'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:12.060181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 888/1784] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:11:11 2022] DELETE /identity/v3/users/5fdb7983e56148528acbd6e24ec3e927 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:12.389471 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73b2d8e-e784-4d66-b075-2e0dd861957a admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:12.391913 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-c73b2d8e-e784-4d66-b075-2e0dd861957a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:12.416961 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73b2d8e-e784-4d66-b075-2e0dd861957a admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:12.444554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:12.496097 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637c7747-430d-4ab2-aec8-e3506d7429e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:12.496743 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c7747-430d-4ab2-aec8-e3506d7429e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:12.497025 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c7747-430d-4ab2-aec8-e3506d7429e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:12.497305 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c7747-430d-4ab2-aec8-e3506d7429e4 admin admin] PATH_INFO: `/v3/projects/1dd734ee08c6447da7b850631a4a8e6a` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:12.498474 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637c7747-430d-4ab2-aec8-e3506d7429e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dd734ee08c6447da7b850631a4a8e6a)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:12.500439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637c7747-430d-4ab2-aec8-e3506d7429e4 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:12.554698 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-637c7747-430d-4ab2-aec8-e3506d7429e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dd734ee08c6447da7b850631a4a8e6a', 'request_id': 'req-637c7747-430d-4ab2-aec8-e3506d7429e4'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:12.560336 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-637c7747-430d-4ab2-aec8-e3506d7429e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dd734ee08c6447da7b850631a4a8e6a', 'request_id': 'req-637c7747-430d-4ab2-aec8-e3506d7429e4'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:12.566682 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-637c7747-430d-4ab2-aec8-e3506d7429e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dd734ee08c6447da7b850631a4a8e6a', 'request_id': 'req-637c7747-430d-4ab2-aec8-e3506d7429e4'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:12.587950 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-637c7747-430d-4ab2-aec8-e3506d7429e4 admin admin] The token cache is being invalidate because project 1dd734ee08c6447da7b850631a4a8e6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:11:12.587950 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-637c7747-430d-4ab2-aec8-e3506d7429e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:12.588874 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-637c7747-430d-4ab2-aec8-e3506d7429e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:12.590554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 897/1785] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:11:12 2022] DELETE /identity/v3/projects/1dd734ee08c6447da7b850631a4a8e6a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:12.595714 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280a772-fdd5-4673-af84-c4c890866e68 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:12.596005 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280a772-fdd5-4673-af84-c4c890866e68 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:12.596283 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280a772-fdd5-4673-af84-c4c890866e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:12.622163 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-3280a772-fdd5-4673-af84-c4c890866e68 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:12.623471 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3280a772-fdd5-4673-af84-c4c890866e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:12.688731 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 889/1786] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:12.697083 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46a60e-1f04-4960-bb30-dd179d9f71db None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:12.697336 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46a60e-1f04-4960-bb30-dd179d9f71db None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:12.697616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46a60e-1f04-4960-bb30-dd179d9f71db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:12.718896 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-4c46a60e-1f04-4960-bb30-dd179d9f71db None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:12.720140 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-4c46a60e-1f04-4960-bb30-dd179d9f71db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:12.728474 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 898/1787] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:12.734648 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3280a772-fdd5-4673-af84-c4c890866e68 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:12.744849 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3280a772-fdd5-4673-af84-c4c890866e68 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:12.755181 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45bfcb1-a16f-4a2d-a4f3-7cc2715b4a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:12.755831 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bfcb1-a16f-4a2d-a4f3-7cc2715b4a2f admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:12.756103 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bfcb1-a16f-4a2d-a4f3-7cc2715b4a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:12.756376 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bfcb1-a16f-4a2d-a4f3-7cc2715b4a2f admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:12.757581 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45bfcb1-a16f-4a2d-a4f3-7cc2715b4a2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:11:12.757944 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45bfcb1-a16f-4a2d-a4f3-7cc2715b4a2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:12.759565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45bfcb1-a16f-4a2d-a4f3-7cc2715b4a2f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:12.774028 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 890/1788] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:11:12.776716 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c46a60e-1f04-4960-bb30-dd179d9f71db None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:12.786088 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c46a60e-1f04-4960-bb30-dd179d9f71db None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:12.791035 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23712d90-8ced-467a-ac36-4e9a869276fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:12.791687 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23712d90-8ced-467a-ac36-4e9a869276fb admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:12.791963 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23712d90-8ced-467a-ac36-4e9a869276fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:12.792257 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23712d90-8ced-467a-ac36-4e9a869276fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:12.792888 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23712d90-8ced-467a-ac36-4e9a869276fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:12.794970 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23712d90-8ced-467a-ac36-4e9a869276fb admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:12.839745 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 899/1789] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:11:12 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:12.841142 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45bfcb1-a16f-4a2d-a4f3-7cc2715b4a2f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:12.851998 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45bfcb1-a16f-4a2d-a4f3-7cc2715b4a2f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:12.920320 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf15391-b335-4799-b985-28a290b83b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:12.923829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf15391-b335-4799-b985-28a290b83b1b admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:12.923829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf15391-b335-4799-b985-28a290b83b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:12.923829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf15391-b335-4799-b985-28a290b83b1b admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:12.923829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf15391-b335-4799-b985-28a290b83b1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:12.924180 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf15391-b335-4799-b985-28a290b83b1b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:12.949935 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 891/1790] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:11:12 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:11:12.951142 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23712d90-8ced-467a-ac36-4e9a869276fb admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:12.961926 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23712d90-8ced-467a-ac36-4e9a869276fb admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:12.969242 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa4eb7-1a5c-41e6-9925-cbcd54e2fedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:12.969874 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa4eb7-1a5c-41e6-9925-cbcd54e2fedc admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:12.970151 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa4eb7-1a5c-41e6-9925-cbcd54e2fedc admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:12.970448 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa4eb7-1a5c-41e6-9925-cbcd54e2fedc admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:12.971617 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa4eb7-1a5c-41e6-9925-cbcd54e2fedc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:12.972972 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa4eb7-1a5c-41e6-9925-cbcd54e2fedc admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:12.991573 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 900/1791] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:12.992902 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf15391-b335-4799-b985-28a290b83b1b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:13.003585 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf15391-b335-4799-b985-28a290b83b1b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:13.008757 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0bf23-ee24-4c17-a184-ae319aa8ba1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:13.009356 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0bf23-ee24-4c17-a184-ae319aa8ba1d admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:13.009631 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0bf23-ee24-4c17-a184-ae319aa8ba1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:13.009903 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0bf23-ee24-4c17-a184-ae319aa8ba1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:13.010541 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac0bf23-ee24-4c17-a184-ae319aa8ba1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:13.012152 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac0bf23-ee24-4c17-a184-ae319aa8ba1d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:13.023331 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 892/1792] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:11:13.027506 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa4eb7-1a5c-41e6-9925-cbcd54e2fedc admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:13.037570 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa4eb7-1a5c-41e6-9925-cbcd54e2fedc admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:13.042350 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8769a61-ef3a-4128-a2a8-2473fa27d49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:13.042993 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8769a61-ef3a-4128-a2a8-2473fa27d49b admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:13.043296 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8769a61-ef3a-4128-a2a8-2473fa27d49b admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:13.043588 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8769a61-ef3a-4128-a2a8-2473fa27d49b admin admin] PATH_INFO: `/v3/projects/53755f69821645b78925fdd5bffe6708/users/d08fdff7d6d94a47a1da5c8b89b08dd5/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:13.058957 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8769a61-ef3a-4128-a2a8-2473fa27d49b admin admin] RBAC: Authorizing `identity:create_grant(project_id=53755f69821645b78925fdd5bffe6708, user_id=d08fdff7d6d94a47a1da5c8b89b08dd5, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:13.061844 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8769a61-ef3a-4128-a2a8-2473fa27d49b admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:13.076707 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 901/1793] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:13 2022] PUT /identity/v3/projects/53755f69821645b78925fdd5bffe6708/users/d08fdff7d6d94a47a1da5c8b89b08dd5/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:13.086956 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6f7a6-272c-4e76-aab3-fe049ecfe780 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:13.087474 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6f7a6-272c-4e76-aab3-fe049ecfe780 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:13.087812 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6f7a6-272c-4e76-aab3-fe049ecfe780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:13.107560 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:13.118628 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-54a6f7a6-272c-4e76-aab3-fe049ecfe780 None None] MFA Rules not processed for user `d08fdff7d6d94a47a1da5c8b89b08dd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:13.120262 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-54a6f7a6-272c-4e76-aab3-fe049ecfe780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:13.198346 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 893/1794] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:13.209805 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8769a61-ef3a-4128-a2a8-2473fa27d49b admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:13.211798 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-e8769a61-ef3a-4128-a2a8-2473fa27d49b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:13.259663 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:13.275433 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8769a61-ef3a-4128-a2a8-2473fa27d49b admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:13.279221 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac350b95-fe09-4b61-babb-50ed33718485 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:13.279843 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac350b95-fe09-4b61-babb-50ed33718485 service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:13.281261 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac350b95-fe09-4b61-babb-50ed33718485 service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:13.281962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac350b95-fe09-4b61-babb-50ed33718485 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:13.297418 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac350b95-fe09-4b61-babb-50ed33718485 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:13.301087 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac350b95-fe09-4b61-babb-50ed33718485 service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:13.314131 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 902/1795] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:11:14.981990 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a6f7a6-272c-4e76-aab3-fe049ecfe780 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:14.984187 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-54a6f7a6-272c-4e76-aab3-fe049ecfe780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:15.155430 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a6f7a6-272c-4e76-aab3-fe049ecfe780 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:15.187368 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe74b1-ec34-44a7-95b5-75ce710ba26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:15.188481 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe74b1-ec34-44a7-95b5-75ce710ba26e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:15.189000 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe74b1-ec34-44a7-95b5-75ce710ba26e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:15.189517 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe74b1-ec34-44a7-95b5-75ce710ba26e admin admin] PATH_INFO: `/v3/users/56aeb18c2d114a42a3a8fb70c95027f6` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:15.191660 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe74b1-ec34-44a7-95b5-75ce710ba26e admin admin] RBAC: Authorizing `identity:delete_user(user_id=56aeb18c2d114a42a3a8fb70c95027f6)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:15.197469 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe74b1-ec34-44a7-95b5-75ce710ba26e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:15.284349 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7dfe74b1-ec34-44a7-95b5-75ce710ba26e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56aeb18c2d114a42a3a8fb70c95027f6'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:15.307178 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7dfe74b1-ec34-44a7-95b5-75ce710ba26e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56aeb18c2d114a42a3a8fb70c95027f6'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:15.307897 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7dfe74b1-ec34-44a7-95b5-75ce710ba26e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56aeb18c2d114a42a3a8fb70c95027f6'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:15.323672 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7dfe74b1-ec34-44a7-95b5-75ce710ba26e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56aeb18c2d114a42a3a8fb70c95027f6'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:15.341488 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7dfe74b1-ec34-44a7-95b5-75ce710ba26e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56aeb18c2d114a42a3a8fb70c95027f6'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:15.348926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 894/1796] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:11:14 2022] DELETE /identity/v3/users/56aeb18c2d114a42a3a8fb70c95027f6 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:15.353530 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3e103-2a48-48c6-bc90-c5258cdc6ce8 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:15.354062 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3e103-2a48-48c6-bc90-c5258cdc6ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:15.354062 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3e103-2a48-48c6-bc90-c5258cdc6ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:15.392694 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-1fd3e103-2a48-48c6-bc90-c5258cdc6ce8 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:15.394681 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-1fd3e103-2a48-48c6-bc90-c5258cdc6ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:15.499225 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 903/1797] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:15.510028 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe74b1-ec34-44a7-95b5-75ce710ba26e admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:15.511724 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7dfe74b1-ec34-44a7-95b5-75ce710ba26e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:15.551638 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:15.631318 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe74b1-ec34-44a7-95b5-75ce710ba26e admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:15.642139 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebdd5be-39a8-4008-a4ec-899d386104e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:15.654050 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdd5be-39a8-4008-a4ec-899d386104e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:15.654050 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdd5be-39a8-4008-a4ec-899d386104e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:15.654050 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdd5be-39a8-4008-a4ec-899d386104e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:15.675982 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebdd5be-39a8-4008-a4ec-899d386104e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:15.680122 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebdd5be-39a8-4008-a4ec-899d386104e7 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:15.714347 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 895/1798] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:11:16.473891 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee4872b-9dc2-4b60-9377-422ced80b9df None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:16.474372 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee4872b-9dc2-4b60-9377-422ced80b9df None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:16.474478 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee4872b-9dc2-4b60-9377-422ced80b9df None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:16.476162 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 904/1799] 10.222.0.23 () {54 vars in 921 bytes} [Thu Oct 13 13:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:11:16.490641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebdd5be-39a8-4008-a4ec-899d386104e7 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:16.492121 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2ebdd5be-39a8-4008-a4ec-899d386104e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:16.504745 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebdd5be-39a8-4008-a4ec-899d386104e7 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:16.510005 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7983f-f576-48f5-9ca1-5b5fe2eb2ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:16.510638 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7983f-f576-48f5-9ca1-5b5fe2eb2ce9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:16.510925 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7983f-f576-48f5-9ca1-5b5fe2eb2ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:16.511312 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7983f-f576-48f5-9ca1-5b5fe2eb2ce9 admin admin] PATH_INFO: `/v3/projects/adf09d575b3844f594773d709f44ff92` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:16.515413 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b7983f-f576-48f5-9ca1-5b5fe2eb2ce9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adf09d575b3844f594773d709f44ff92)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:16.517729 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b7983f-f576-48f5-9ca1-5b5fe2eb2ce9 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:16.525297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0408a8-bf74-4139-882a-6f87583afcf4 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:16.525670 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0408a8-bf74-4139-882a-6f87583afcf4 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:16.526162 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0408a8-bf74-4139-882a-6f87583afcf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:16.548994 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-1a0408a8-bf74-4139-882a-6f87583afcf4 None None] MFA Rules not processed for user `1a971cdb7f9d4e6692c6326c103d7303`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:16.552521 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-1a0408a8-bf74-4139-882a-6f87583afcf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:16.600833 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-79b7983f-f576-48f5-9ca1-5b5fe2eb2ce9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adf09d575b3844f594773d709f44ff92', 'request_id': 'req-79b7983f-f576-48f5-9ca1-5b5fe2eb2ce9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:16.610423 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-79b7983f-f576-48f5-9ca1-5b5fe2eb2ce9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adf09d575b3844f594773d709f44ff92', 'request_id': 'req-79b7983f-f576-48f5-9ca1-5b5fe2eb2ce9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:16.619746 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-79b7983f-f576-48f5-9ca1-5b5fe2eb2ce9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adf09d575b3844f594773d709f44ff92', 'request_id': 'req-79b7983f-f576-48f5-9ca1-5b5fe2eb2ce9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:16.633761 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-79b7983f-f576-48f5-9ca1-5b5fe2eb2ce9 admin admin] The token cache is being invalidate because project adf09d575b3844f594773d709f44ff92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:11:16.634071 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-79b7983f-f576-48f5-9ca1-5b5fe2eb2ce9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:16.634794 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-79b7983f-f576-48f5-9ca1-5b5fe2eb2ce9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:16.636925 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 896/1800] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:11:16 2022] DELETE /identity/v3/projects/adf09d575b3844f594773d709f44ff92 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:16.638035 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7983f-f576-48f5-9ca1-5b5fe2eb2ce9 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:16.639655 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-79b7983f-f576-48f5-9ca1-5b5fe2eb2ce9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:16.662316 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 905/1801] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Oct 13 13:11:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:11:16.663576 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0408a8-bf74-4139-882a-6f87583afcf4 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:16.665201 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-1a0408a8-bf74-4139-882a-6f87583afcf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:16.681581 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0408a8-bf74-4139-882a-6f87583afcf4 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:16.711282 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7983f-f576-48f5-9ca1-5b5fe2eb2ce9 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:16.716838 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d75224d-3b53-47a4-8d6f-fa22dc8855b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:16.717492 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75224d-3b53-47a4-8d6f-fa22dc8855b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:16.717896 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75224d-3b53-47a4-8d6f-fa22dc8855b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:16.718177 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75224d-3b53-47a4-8d6f-fa22dc8855b2 admin admin] PATH_INFO: `/v3/users/e3a049e77aba470db21e26d1bc8caa80` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:16.719344 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d75224d-3b53-47a4-8d6f-fa22dc8855b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3a049e77aba470db21e26d1bc8caa80)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:16.721258 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d75224d-3b53-47a4-8d6f-fa22dc8855b2 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:16.738323 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aefdd47-be9c-4245-b529-e64292f455f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:16.738995 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefdd47-be9c-4245-b529-e64292f455f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:16.739314 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefdd47-be9c-4245-b529-e64292f455f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:16.739588 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefdd47-be9c-4245-b529-e64292f455f5 admin admin] PATH_INFO: `/v3/users/ca0698594c13415a98167b73e36931a9` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:16.740705 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aefdd47-be9c-4245-b529-e64292f455f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca0698594c13415a98167b73e36931a9)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:16.743329 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aefdd47-be9c-4245-b529-e64292f455f5 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:16.771707 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:16.771935 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:16.773767 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1d75224d-3b53-47a4-8d6f-fa22dc8855b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3a049e77aba470db21e26d1bc8caa80'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:16.787302 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1d75224d-3b53-47a4-8d6f-fa22dc8855b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3a049e77aba470db21e26d1bc8caa80'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:16.788508 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1d75224d-3b53-47a4-8d6f-fa22dc8855b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3a049e77aba470db21e26d1bc8caa80'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:16.801275 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1d75224d-3b53-47a4-8d6f-fa22dc8855b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3a049e77aba470db21e26d1bc8caa80'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:16.813385 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-6aefdd47-be9c-4245-b529-e64292f455f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca0698594c13415a98167b73e36931a9'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:16.827701 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1d75224d-3b53-47a4-8d6f-fa22dc8855b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3a049e77aba470db21e26d1bc8caa80'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:16.828566 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-6aefdd47-be9c-4245-b529-e64292f455f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca0698594c13415a98167b73e36931a9'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:16.830278 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-6aefdd47-be9c-4245-b529-e64292f455f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca0698594c13415a98167b73e36931a9'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:16.832033 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-6aefdd47-be9c-4245-b529-e64292f455f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca0698594c13415a98167b73e36931a9'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:16.837206 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 897/1802] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:11:16 2022] DELETE /identity/v3/users/e3a049e77aba470db21e26d1bc8caa80 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:16.843927 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b90251-b2f4-4d6a-abb9-d09f65adf6ab None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:16.844321 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b90251-b2f4-4d6a-abb9-d09f65adf6ab None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:16.844733 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b90251-b2f4-4d6a-abb9-d09f65adf6ab None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:16.846211 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 898/1803] 10.222.0.23 () {54 vars in 961 bytes} [Thu Oct 13 13:11:16 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:11:16.852415 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-6aefdd47-be9c-4245-b529-e64292f455f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca0698594c13415a98167b73e36931a9'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:16.859147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85bb53-905c-4d4d-9bea-449df7bdaa2d None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:16.859574 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85bb53-905c-4d4d-9bea-449df7bdaa2d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:16.859928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85bb53-905c-4d4d-9bea-449df7bdaa2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:16.883470 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 906/1804] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:11:16 2022] DELETE /identity/v3/users/ca0698594c13415a98167b73e36931a9 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:16.894780 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8b85bb53-905c-4d4d-9bea-449df7bdaa2d None None] MFA Rules not processed for user `1a971cdb7f9d4e6692c6326c103d7303`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:16.896218 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8b85bb53-905c-4d4d-9bea-449df7bdaa2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:16.930645 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aefdd47-be9c-4245-b529-e64292f455f5 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:16.932321 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-6aefdd47-be9c-4245-b529-e64292f455f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:16.997680 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 899/1805] 10.222.0.23 () {58 vars in 1058 bytes} [Thu Oct 13 13:11:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:17.009244 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85bb53-905c-4d4d-9bea-449df7bdaa2d None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:17.024363 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85bb53-905c-4d4d-9bea-449df7bdaa2d None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:17.030006 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e34446-169a-4ce9-90a9-069d259e0a44 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a971cdb7f9d4e6692c6326c103d7303', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:17.030868 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e34446-169a-4ce9-90a9-069d259e0a44 service placement] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:17.031028 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e34446-169a-4ce9-90a9-069d259e0a44 service placement] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:17.031338 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e34446-169a-4ce9-90a9-069d259e0a44 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:17.033234 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-51e34446-169a-4ce9-90a9-069d259e0a44 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:17.034154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aefdd47-be9c-4245-b529-e64292f455f5 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:17.043873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a398c-f6d5-49ca-a086-6c4dd4a77ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:17.045687 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a398c-f6d5-49ca-a086-6c4dd4a77ec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:17.047188 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a398c-f6d5-49ca-a086-6c4dd4a77ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:17.047764 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e34446-169a-4ce9-90a9-069d259e0a44 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:17.047994 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a398c-f6d5-49ca-a086-6c4dd4a77ec9 admin admin] PATH_INFO: `/v3/users/d08fdff7d6d94a47a1da5c8b89b08dd5` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:17.049357 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e34446-169a-4ce9-90a9-069d259e0a44 service placement] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:17.049982 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878a398c-f6d5-49ca-a086-6c4dd4a77ec9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d08fdff7d6d94a47a1da5c8b89b08dd5)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:17.053170 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878a398c-f6d5-49ca-a086-6c4dd4a77ec9 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:17.062851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 900/1806] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 13 13:11:17.120714 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-878a398c-f6d5-49ca-a086-6c4dd4a77ec9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd08fdff7d6d94a47a1da5c8b89b08dd5'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:17.130921 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-878a398c-f6d5-49ca-a086-6c4dd4a77ec9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd08fdff7d6d94a47a1da5c8b89b08dd5'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:17.131742 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-878a398c-f6d5-49ca-a086-6c4dd4a77ec9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd08fdff7d6d94a47a1da5c8b89b08dd5'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:17.132394 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-878a398c-f6d5-49ca-a086-6c4dd4a77ec9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd08fdff7d6d94a47a1da5c8b89b08dd5'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:17.141962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-878a398c-f6d5-49ca-a086-6c4dd4a77ec9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd08fdff7d6d94a47a1da5c8b89b08dd5'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:17.168108 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 907/1807] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:11:16 2022] DELETE /identity/v3/users/d08fdff7d6d94a47a1da5c8b89b08dd5 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:17.169772 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0a2ab-9103-4c00-83d1-d4e7ca1d1bb9 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:17.170062 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0a2ab-9103-4c00-83d1-d4e7ca1d1bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:17.170339 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0a2ab-9103-4c00-83d1-d4e7ca1d1bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:17.196437 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-46b0a2ab-9103-4c00-83d1-d4e7ca1d1bb9 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:17.196437 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-46b0a2ab-9103-4c00-83d1-d4e7ca1d1bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:17.233983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a398c-f6d5-49ca-a086-6c4dd4a77ec9 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:17.236038 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-878a398c-f6d5-49ca-a086-6c4dd4a77ec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:17.262547 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:17.283513 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 901/1808] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:17.296870 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b0a2ab-9103-4c00-83d1-d4e7ca1d1bb9 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:17.298259 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-46b0a2ab-9103-4c00-83d1-d4e7ca1d1bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:17.317035 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a398c-f6d5-49ca-a086-6c4dd4a77ec9 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:17.323632 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc02631-c61e-4b4c-a3b6-1a0cc6ee131c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:17.324282 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc02631-c61e-4b4c-a3b6-1a0cc6ee131c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:17.324560 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc02631-c61e-4b4c-a3b6-1a0cc6ee131c admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:17.324830 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc02631-c61e-4b4c-a3b6-1a0cc6ee131c admin admin] PATH_INFO: `/v3/projects/6027a602300d4d2bbf33ac329b2960f6` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:17.326280 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc02631-c61e-4b4c-a3b6-1a0cc6ee131c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6027a602300d4d2bbf33ac329b2960f6)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:17.327952 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc02631-c61e-4b4c-a3b6-1a0cc6ee131c admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:17.357568 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:17.367724 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b0a2ab-9103-4c00-83d1-d4e7ca1d1bb9 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:17.392799 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-9cc02631-c61e-4b4c-a3b6-1a0cc6ee131c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6027a602300d4d2bbf33ac329b2960f6', 'request_id': 'req-9cc02631-c61e-4b4c-a3b6-1a0cc6ee131c'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:17.399601 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-9cc02631-c61e-4b4c-a3b6-1a0cc6ee131c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6027a602300d4d2bbf33ac329b2960f6', 'request_id': 'req-9cc02631-c61e-4b4c-a3b6-1a0cc6ee131c'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:17.411329 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-9cc02631-c61e-4b4c-a3b6-1a0cc6ee131c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6027a602300d4d2bbf33ac329b2960f6', 'request_id': 'req-9cc02631-c61e-4b4c-a3b6-1a0cc6ee131c'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:17.430615 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-9cc02631-c61e-4b4c-a3b6-1a0cc6ee131c admin admin] The token cache is being invalidate because project 6027a602300d4d2bbf33ac329b2960f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:11:17.430906 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-9cc02631-c61e-4b4c-a3b6-1a0cc6ee131c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:17.431597 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-9cc02631-c61e-4b4c-a3b6-1a0cc6ee131c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:17.433112 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 908/1809] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:11:17 2022] DELETE /identity/v3/projects/6027a602300d4d2bbf33ac329b2960f6 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:17.437302 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc02631-c61e-4b4c-a3b6-1a0cc6ee131c admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:17.438629 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-9cc02631-c61e-4b4c-a3b6-1a0cc6ee131c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:17.446158 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264aa48-6a76-499c-a8cf-0861e95d0b73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:17.446857 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264aa48-6a76-499c-a8cf-0861e95d0b73 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:17.447242 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264aa48-6a76-499c-a8cf-0861e95d0b73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:17.447585 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264aa48-6a76-499c-a8cf-0861e95d0b73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:17.449530 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3264aa48-6a76-499c-a8cf-0861e95d0b73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:17.505352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc02631-c61e-4b4c-a3b6-1a0cc6ee131c admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:17.512001 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be23ac56-fc8a-47cd-bdd1-e5091687bde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:17.512656 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23ac56-fc8a-47cd-bdd1-e5091687bde5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:17.513107 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23ac56-fc8a-47cd-bdd1-e5091687bde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:17.513443 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23ac56-fc8a-47cd-bdd1-e5091687bde5 admin admin] PATH_INFO: `/v3/projects/df202d6d34a14d03acc9afd3b6ad1c25` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:17.514757 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be23ac56-fc8a-47cd-bdd1-e5091687bde5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df202d6d34a14d03acc9afd3b6ad1c25)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:17.516725 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be23ac56-fc8a-47cd-bdd1-e5091687bde5 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:17.518086 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3264aa48-6a76-499c-a8cf-0861e95d0b73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:17.519855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3264aa48-6a76-499c-a8cf-0861e95d0b73 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:17.530379 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 902/1810] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 13 13:11:17.536954 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3faabc-4767-421f-ba47-96d12947d0bc None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:17.538432 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3faabc-4767-421f-ba47-96d12947d0bc None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:17.538831 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3faabc-4767-421f-ba47-96d12947d0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:17.564466 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-2c3faabc-4767-421f-ba47-96d12947d0bc None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:17.566075 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2c3faabc-4767-421f-ba47-96d12947d0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:17.574134 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-be23ac56-fc8a-47cd-bdd1-e5091687bde5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df202d6d34a14d03acc9afd3b6ad1c25', 'request_id': 'req-be23ac56-fc8a-47cd-bdd1-e5091687bde5'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:17.576363 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:17.581488 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-be23ac56-fc8a-47cd-bdd1-e5091687bde5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df202d6d34a14d03acc9afd3b6ad1c25', 'request_id': 'req-be23ac56-fc8a-47cd-bdd1-e5091687bde5'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:17.588845 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-be23ac56-fc8a-47cd-bdd1-e5091687bde5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df202d6d34a14d03acc9afd3b6ad1c25', 'request_id': 'req-be23ac56-fc8a-47cd-bdd1-e5091687bde5'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:17.607144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-be23ac56-fc8a-47cd-bdd1-e5091687bde5 admin admin] The token cache is being invalidate because project df202d6d34a14d03acc9afd3b6ad1c25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:11:17.607991 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-be23ac56-fc8a-47cd-bdd1-e5091687bde5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:17.609068 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-be23ac56-fc8a-47cd-bdd1-e5091687bde5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:17.611329 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 909/1811] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:11:17 2022] DELETE /identity/v3/projects/df202d6d34a14d03acc9afd3b6ad1c25 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:17.617476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0969b81-53b1-41c1-8dd4-8d1eb6d99364 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:17.617876 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0969b81-53b1-41c1-8dd4-8d1eb6d99364 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:17.618248 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0969b81-53b1-41c1-8dd4-8d1eb6d99364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:17.649720 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-c0969b81-53b1-41c1-8dd4-8d1eb6d99364 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:17.655269 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-c0969b81-53b1-41c1-8dd4-8d1eb6d99364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:17.663504 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 903/1812] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:17.666770 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63edfd-51f0-4c91-9c44-123ed0efff2a None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:17.667309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63edfd-51f0-4c91-9c44-123ed0efff2a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:17.667725 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63edfd-51f0-4c91-9c44-123ed0efff2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:17.679020 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 910/1813] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:11:17.680930 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f7da1-bcff-4aee-9273-924397f4f438 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:17.682822 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f7da1-bcff-4aee-9273-924397f4f438 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:17.683159 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f7da1-bcff-4aee-9273-924397f4f438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:17.702152 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-9b63edfd-51f0-4c91-9c44-123ed0efff2a None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:17.703439 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9b63edfd-51f0-4c91-9c44-123ed0efff2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:17.712743 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-c54f7da1-bcff-4aee-9273-924397f4f438 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:17.716226 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 904/1814] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:17.717730 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-c54f7da1-bcff-4aee-9273-924397f4f438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:17.718424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b63edfd-51f0-4c91-9c44-123ed0efff2a None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:17.730757 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b63edfd-51f0-4c91-9c44-123ed0efff2a None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:17.734948 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 911/1815] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:17.738411 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f7da1-bcff-4aee-9273-924397f4f438 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:17.740034 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40199846-3f88-420c-ab58-004832f57bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:17.741290 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40199846-3f88-420c-ab58-004832f57bff admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:17.741875 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40199846-3f88-420c-ab58-004832f57bff admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:17.742262 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40199846-3f88-420c-ab58-004832f57bff admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:17.743024 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40199846-3f88-420c-ab58-004832f57bff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:11:17.743459 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40199846-3f88-420c-ab58-004832f57bff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:17.745224 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40199846-3f88-420c-ab58-004832f57bff admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:17.752995 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f7da1-bcff-4aee-9273-924397f4f438 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:17.761456 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 905/1816] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:11:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:11:17.766747 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6599da-5527-4f83-bd99-95414511d1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:17.767608 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6599da-5527-4f83-bd99-95414511d1ed admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:17.767895 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6599da-5527-4f83-bd99-95414511d1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:17.768168 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6599da-5527-4f83-bd99-95414511d1ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:17.768463 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40199846-3f88-420c-ab58-004832f57bff admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:17.768836 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6599da-5527-4f83-bd99-95414511d1ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:11:17.769190 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6599da-5527-4f83-bd99-95414511d1ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:17.770809 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6599da-5527-4f83-bd99-95414511d1ed admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:17.783695 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40199846-3f88-420c-ab58-004832f57bff admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:17.790500 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 912/1817] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:11:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:11:17.794689 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6599da-5527-4f83-bd99-95414511d1ed admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:17.795495 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd7e8a-c858-4464-b8da-798daa105112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:17.796256 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd7e8a-c858-4464-b8da-798daa105112 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:17.796641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd7e8a-c858-4464-b8da-798daa105112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:17.797014 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfd7e8a-c858-4464-b8da-798daa105112 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:17.797754 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfd7e8a-c858-4464-b8da-798daa105112 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:17.799967 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfd7e8a-c858-4464-b8da-798daa105112 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:17.807240 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6599da-5527-4f83-bd99-95414511d1ed admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:17.812777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be300c0-f2ad-44ba-a0c1-8bf47cbcc7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:17.813696 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be300c0-f2ad-44ba-a0c1-8bf47cbcc7dc admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:17.814116 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be300c0-f2ad-44ba-a0c1-8bf47cbcc7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:17.814525 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be300c0-f2ad-44ba-a0c1-8bf47cbcc7dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:17.815352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be300c0-f2ad-44ba-a0c1-8bf47cbcc7dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:17.817489 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be300c0-f2ad-44ba-a0c1-8bf47cbcc7dc admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:17.836945 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 906/1818] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:11:17 2022] POST /identity/v3/projects => generated 373 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:17.842017 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd7e8a-c858-4464-b8da-798daa105112 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:17.860481 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 913/1819] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:11:17 2022] POST /identity/v3/projects => generated 407 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:11:17.861435 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be300c0-f2ad-44ba-a0c1-8bf47cbcc7dc admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:17.889385 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:17.964108 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be300c0-f2ad-44ba-a0c1-8bf47cbcc7dc admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:17.966879 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfd7e8a-c858-4464-b8da-798daa105112 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:17.969015 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47fa64-b5d7-40e5-a12b-72db4f0d7db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:17.969727 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47fa64-b5d7-40e5-a12b-72db4f0d7db5 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:17.970072 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47fa64-b5d7-40e5-a12b-72db4f0d7db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:17.970354 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47fa64-b5d7-40e5-a12b-72db4f0d7db5 admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:17.971035 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c47fa64-b5d7-40e5-a12b-72db4f0d7db5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:17.971914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d080e54-cf4a-4eaa-b85c-eeb29ac269dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:17.972697 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d080e54-cf4a-4eaa-b85c-eeb29ac269dc admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:17.972989 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c47fa64-b5d7-40e5-a12b-72db4f0d7db5 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:17.973091 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d080e54-cf4a-4eaa-b85c-eeb29ac269dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:17.973435 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d080e54-cf4a-4eaa-b85c-eeb29ac269dc admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:17.974131 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d080e54-cf4a-4eaa-b85c-eeb29ac269dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:17.976671 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d080e54-cf4a-4eaa-b85c-eeb29ac269dc admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:17.999096 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 914/1820] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:11:17 2022] POST /identity/v3/users => generated 304 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:11:17.999096 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47fa64-b5d7-40e5-a12b-72db4f0d7db5 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:17.999096 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-8c47fa64-b5d7-40e5-a12b-72db4f0d7db5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:18.004380 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 907/1821] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:11:17 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:11:18.004380 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d080e54-cf4a-4eaa-b85c-eeb29ac269dc admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:18.012327 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d080e54-cf4a-4eaa-b85c-eeb29ac269dc admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:18.012519 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47fa64-b5d7-40e5-a12b-72db4f0d7db5 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:18.017422 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb675a5-fed5-4183-b4cf-165b1032808f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:18.018065 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb675a5-fed5-4183-b4cf-165b1032808f admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:18.018349 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb675a5-fed5-4183-b4cf-165b1032808f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:18.018620 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb675a5-fed5-4183-b4cf-165b1032808f admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:18.018807 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eba0bc-c09e-42d9-8d22-c072467df744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:18.019331 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb675a5-fed5-4183-b4cf-165b1032808f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:18.019656 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eba0bc-c09e-42d9-8d22-c072467df744 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:18.020593 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eba0bc-c09e-42d9-8d22-c072467df744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:18.020967 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eba0bc-c09e-42d9-8d22-c072467df744 admin admin] PATH_INFO: `/v3/projects/53755f69821645b78925fdd5bffe6708` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:18.021059 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb675a5-fed5-4183-b4cf-165b1032808f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:18.022331 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eba0bc-c09e-42d9-8d22-c072467df744 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53755f69821645b78925fdd5bffe6708)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:18.025104 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eba0bc-c09e-42d9-8d22-c072467df744 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:18.032751 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 908/1822] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:18.036507 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb675a5-fed5-4183-b4cf-165b1032808f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:18.046465 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb675a5-fed5-4183-b4cf-165b1032808f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:18.052348 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628bf3d4-de38-4779-afec-a090486ef627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:18.052991 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628bf3d4-de38-4779-afec-a090486ef627 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:18.053922 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628bf3d4-de38-4779-afec-a090486ef627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:18.054200 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628bf3d4-de38-4779-afec-a090486ef627 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:18.055344 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628bf3d4-de38-4779-afec-a090486ef627 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:18.057132 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628bf3d4-de38-4779-afec-a090486ef627 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:18.067995 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 909/1823] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:18.068688 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628bf3d4-de38-4779-afec-a090486ef627 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:18.076769 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-31eba0bc-c09e-42d9-8d22-c072467df744 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53755f69821645b78925fdd5bffe6708', 'request_id': 'req-31eba0bc-c09e-42d9-8d22-c072467df744'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:18.078466 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628bf3d4-de38-4779-afec-a090486ef627 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:18.083147 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-31eba0bc-c09e-42d9-8d22-c072467df744 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53755f69821645b78925fdd5bffe6708', 'request_id': 'req-31eba0bc-c09e-42d9-8d22-c072467df744'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:18.093665 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-31eba0bc-c09e-42d9-8d22-c072467df744 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53755f69821645b78925fdd5bffe6708', 'request_id': 'req-31eba0bc-c09e-42d9-8d22-c072467df744'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:18.107101 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-31eba0bc-c09e-42d9-8d22-c072467df744 admin admin] The token cache is being invalidate because project 53755f69821645b78925fdd5bffe6708 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:11:18.107474 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-31eba0bc-c09e-42d9-8d22-c072467df744 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:18.108211 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-31eba0bc-c09e-42d9-8d22-c072467df744 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:18.109777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 915/1824] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:11:17 2022] DELETE /identity/v3/projects/53755f69821645b78925fdd5bffe6708 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:18.110696 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eba0bc-c09e-42d9-8d22-c072467df744 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:18.113416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-31eba0bc-c09e-42d9-8d22-c072467df744 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:18.121024 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=85,keystone:DELETE=9,keystone:INSERT=6 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:18.173842 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f55a32-f7a0-4f1e-9150-1b1b1ec4ce71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:18.174564 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f55a32-f7a0-4f1e-9150-1b1b1ec4ce71 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:18.174907 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f55a32-f7a0-4f1e-9150-1b1b1ec4ce71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:18.175287 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f55a32-f7a0-4f1e-9150-1b1b1ec4ce71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:18.175951 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f55a32-f7a0-4f1e-9150-1b1b1ec4ce71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:18.177849 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f55a32-f7a0-4f1e-9150-1b1b1ec4ce71 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:18.189834 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 910/1825] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:18.194418 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f55a32-f7a0-4f1e-9150-1b1b1ec4ce71 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:18.196231 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b6f55a32-f7a0-4f1e-9150-1b1b1ec4ce71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:18.212993 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f55a32-f7a0-4f1e-9150-1b1b1ec4ce71 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:18.213580 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eba0bc-c09e-42d9-8d22-c072467df744 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:18.218002 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103b9d2-b50d-4af1-9e67-1ef769beb771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:18.218685 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103b9d2-b50d-4af1-9e67-1ef769beb771 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:18.218973 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103b9d2-b50d-4af1-9e67-1ef769beb771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:18.219261 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103b9d2-b50d-4af1-9e67-1ef769beb771 admin admin] PATH_INFO: `/v3/projects/f92874b7f58c479db71250dc19c31251/users/40be4afcbd5e49fa85f596cce44026cd/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:18.227597 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a777108d-bfcf-4050-a559-0ce14180344c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:18.228821 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777108d-bfcf-4050-a559-0ce14180344c admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:18.229208 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777108d-bfcf-4050-a559-0ce14180344c admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:18.229588 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777108d-bfcf-4050-a559-0ce14180344c admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:18.230811 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a777108d-bfcf-4050-a559-0ce14180344c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:18.234850 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a777108d-bfcf-4050-a559-0ce14180344c admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:18.256141 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6103b9d2-b50d-4af1-9e67-1ef769beb771 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f92874b7f58c479db71250dc19c31251, user_id=40be4afcbd5e49fa85f596cce44026cd, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:18.256737 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 916/1826] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:18.258519 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a777108d-bfcf-4050-a559-0ce14180344c admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:18.267891 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6103b9d2-b50d-4af1-9e67-1ef769beb771 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:18.279171 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a777108d-bfcf-4050-a559-0ce14180344c admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:18.284286 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b34e2-cf83-48e3-868b-6d927f4c6bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:18.284970 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b34e2-cf83-48e3-868b-6d927f4c6bfa admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:18.285268 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b34e2-cf83-48e3-868b-6d927f4c6bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:18.285589 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b34e2-cf83-48e3-868b-6d927f4c6bfa admin admin] PATH_INFO: `/v3/projects/88b27ae498a74fd0aca979f928cf1742/users/e552ea32209f4518b7dbe0607ea217f4/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:18.289094 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 911/1827] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:18 2022] PUT /identity/v3/projects/f92874b7f58c479db71250dc19c31251/users/40be4afcbd5e49fa85f596cce44026cd/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:11:18.300609 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d8620-d6c7-4dee-9967-4f0fbb29b1bf None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:18.300963 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d8620-d6c7-4dee-9967-4f0fbb29b1bf None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:18.301587 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d8620-d6c7-4dee-9967-4f0fbb29b1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:18.304044 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b34e2-cf83-48e3-868b-6d927f4c6bfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=88b27ae498a74fd0aca979f928cf1742, user_id=e552ea32209f4518b7dbe0607ea217f4, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:18.306794 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b34e2-cf83-48e3-868b-6d927f4c6bfa admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:18.321471 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 917/1828] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:18 2022] PUT /identity/v3/projects/88b27ae498a74fd0aca979f928cf1742/users/e552ea32209f4518b7dbe0607ea217f4/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:18.331030 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-693d8620-d6c7-4dee-9967-4f0fbb29b1bf None None] MFA Rules not processed for user `40be4afcbd5e49fa85f596cce44026cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:18.331575 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e893fc5-e32f-4bee-9e2b-6d10b0df9484 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:18.331858 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e893fc5-e32f-4bee-9e2b-6d10b0df9484 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:18.332125 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e893fc5-e32f-4bee-9e2b-6d10b0df9484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:18.332373 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-693d8620-d6c7-4dee-9967-4f0fbb29b1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:18.367827 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-4e893fc5-e32f-4bee-9e2b-6d10b0df9484 None None] MFA Rules not processed for user `e552ea32209f4518b7dbe0607ea217f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:18.368961 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-4e893fc5-e32f-4bee-9e2b-6d10b0df9484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:18.374623 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:18.400046 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 912/1829] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:18 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:11:18.404277 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d8620-d6c7-4dee-9967-4f0fbb29b1bf None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:18.447150 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 918/1830] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:11:18.451572 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e893fc5-e32f-4bee-9e2b-6d10b0df9484 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:18.469699 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d8620-d6c7-4dee-9967-4f0fbb29b1bf None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:18.474344 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4206264-3f65-4ea4-ad4f-e7f9f2f540dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:18.474946 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4206264-3f65-4ea4-ad4f-e7f9f2f540dc admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:18.475810 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4206264-3f65-4ea4-ad4f-e7f9f2f540dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:18.476138 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4206264-3f65-4ea4-ad4f-e7f9f2f540dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:18.476771 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4206264-3f65-4ea4-ad4f-e7f9f2f540dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:18.478671 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4206264-3f65-4ea4-ad4f-e7f9f2f540dc admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:18.507109 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 913/1831] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:11:18 2022] POST /identity/v3/projects => generated 409 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:11:18.510807 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4206264-3f65-4ea4-ad4f-e7f9f2f540dc admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:18.523265 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e893fc5-e32f-4bee-9e2b-6d10b0df9484 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:18.532123 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:18.572445 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4206264-3f65-4ea4-ad4f-e7f9f2f540dc admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:18.580514 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0e6ce-d5c4-457e-84ce-36b6b813c1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:18.580514 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0e6ce-d5c4-457e-84ce-36b6b813c1fc admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:18.580514 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0e6ce-d5c4-457e-84ce-36b6b813c1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:18.580514 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0e6ce-d5c4-457e-84ce-36b6b813c1fc admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:18.580963 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d0e6ce-d5c4-457e-84ce-36b6b813c1fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:18.581883 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8441179-c7c2-4873-ad59-8b6e793a6080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:18.582134 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d0e6ce-d5c4-457e-84ce-36b6b813c1fc admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:18.582543 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8441179-c7c2-4873-ad59-8b6e793a6080 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:18.582822 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8441179-c7c2-4873-ad59-8b6e793a6080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:18.583134 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8441179-c7c2-4873-ad59-8b6e793a6080 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:18.583802 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8441179-c7c2-4873-ad59-8b6e793a6080 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:18.586404 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8441179-c7c2-4873-ad59-8b6e793a6080 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:18.605441 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 914/1832] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:11:18 2022] POST /identity/v3/users => generated 311 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:11:18.609390 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0e6ce-d5c4-457e-84ce-36b6b813c1fc admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:18.617217 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 919/1833] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:11:18 2022] POST /identity/v3/projects => generated 371 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:18.620938 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8441179-c7c2-4873-ad59-8b6e793a6080 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:18.621264 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0e6ce-d5c4-457e-84ce-36b6b813c1fc admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:18.673837 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:18.681316 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c5f21-285a-4374-9f4d-fda033e5eafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:18.682141 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c5f21-285a-4374-9f4d-fda033e5eafa admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:18.682427 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c5f21-285a-4374-9f4d-fda033e5eafa admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:18.682703 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c5f21-285a-4374-9f4d-fda033e5eafa admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:18.683381 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c5f21-285a-4374-9f4d-fda033e5eafa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:18.684962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8441179-c7c2-4873-ad59-8b6e793a6080 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:18.685082 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c5f21-285a-4374-9f4d-fda033e5eafa admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:18.689579 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65a3f8-6073-4c8b-8486-0923390dc5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:18.690162 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65a3f8-6073-4c8b-8486-0923390dc5aa admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:18.690442 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65a3f8-6073-4c8b-8486-0923390dc5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:18.690727 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65a3f8-6073-4c8b-8486-0923390dc5aa admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:18.691363 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed65a3f8-6073-4c8b-8486-0923390dc5aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:18.693277 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed65a3f8-6073-4c8b-8486-0923390dc5aa admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:18.695555 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 915/1834] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:11:18.699626 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c5f21-285a-4374-9f4d-fda033e5eafa admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:18.710547 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c5f21-285a-4374-9f4d-fda033e5eafa admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:18.720243 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd874de-73f9-4dd7-a517-accb111dda9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:18.720920 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd874de-73f9-4dd7-a517-accb111dda9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:18.721249 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd874de-73f9-4dd7-a517-accb111dda9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:18.721477 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd874de-73f9-4dd7-a517-accb111dda9a admin admin] PATH_INFO: `/v3/projects/7504520e70d345bf930e89e385957869/users/bf0be741f0cc402e93251339a7ca98ed/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:18.724602 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 920/1835] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:11:18 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:11:18.727008 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65a3f8-6073-4c8b-8486-0923390dc5aa admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:18.736520 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd874de-73f9-4dd7-a517-accb111dda9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7504520e70d345bf930e89e385957869, user_id=bf0be741f0cc402e93251339a7ca98ed, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:18.736829 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65a3f8-6073-4c8b-8486-0923390dc5aa admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:18.739230 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd874de-73f9-4dd7-a517-accb111dda9a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:18.741419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16833ce-840f-4162-a204-c46376d80233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:18.742003 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16833ce-840f-4162-a204-c46376d80233 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:18.742281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16833ce-840f-4162-a204-c46376d80233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:18.742695 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16833ce-840f-4162-a204-c46376d80233 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:18.743376 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16833ce-840f-4162-a204-c46376d80233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:18.744991 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16833ce-840f-4162-a204-c46376d80233 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:18.752508 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 916/1836] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:18 2022] PUT /identity/v3/projects/7504520e70d345bf930e89e385957869/users/bf0be741f0cc402e93251339a7ca98ed/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:18.756966 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd874de-73f9-4dd7-a517-accb111dda9a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:18.757271 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 921/1837] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:11:18.760737 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16833ce-840f-4162-a204-c46376d80233 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:18.770195 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16833ce-840f-4162-a204-c46376d80233 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:18.820243 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd874de-73f9-4dd7-a517-accb111dda9a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:18.828709 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b15910-4144-4ee3-b935-eec139007667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:18.829461 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b15910-4144-4ee3-b935-eec139007667 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:18.829686 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db20f8b1-c9f4-4c95-9917-667ed8ac1a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:18.829805 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b15910-4144-4ee3-b935-eec139007667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:18.830007 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b15910-4144-4ee3-b935-eec139007667 admin admin] PATH_INFO: `/v3/projects/4fdbb347b6fc405e983f5b456f5e0bb2/users/695df8101cb94060a488840999d05fa2/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:18.830306 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20f8b1-c9f4-4c95-9917-667ed8ac1a73 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:18.830590 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20f8b1-c9f4-4c95-9917-667ed8ac1a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:18.830862 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20f8b1-c9f4-4c95-9917-667ed8ac1a73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:18.831549 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db20f8b1-c9f4-4c95-9917-667ed8ac1a73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:18.833376 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db20f8b1-c9f4-4c95-9917-667ed8ac1a73 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:18.844511 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 917/1838] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:11:18.845597 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b15910-4144-4ee3-b935-eec139007667 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fdbb347b6fc405e983f5b456f5e0bb2, user_id=695df8101cb94060a488840999d05fa2, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:18.848496 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b15910-4144-4ee3-b935-eec139007667 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:18.848629 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db20f8b1-c9f4-4c95-9917-667ed8ac1a73 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:18.857328 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:18.858566 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db20f8b1-c9f4-4c95-9917-667ed8ac1a73 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:18.868693 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 922/1839] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:18 2022] PUT /identity/v3/projects/4fdbb347b6fc405e983f5b456f5e0bb2/users/695df8101cb94060a488840999d05fa2/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:18.868693 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b15910-4144-4ee3-b935-eec139007667 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:18.936658 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f99a135-ef38-4838-911d-889542aca515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:18.937804 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99a135-ef38-4838-911d-889542aca515 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:18.938095 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99a135-ef38-4838-911d-889542aca515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:18.938371 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99a135-ef38-4838-911d-889542aca515 admin admin] PATH_INFO: `/v3/projects/7504520e70d345bf930e89e385957869/users/bf0be741f0cc402e93251339a7ca98ed/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:18.940377 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f99a135-ef38-4838-911d-889542aca515 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7504520e70d345bf930e89e385957869, user_id=bf0be741f0cc402e93251339a7ca98ed, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:18.943237 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b15910-4144-4ee3-b935-eec139007667 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:18.943385 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f99a135-ef38-4838-911d-889542aca515 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:18.949532 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16412fc3-67f2-4359-aad8-3a02e4c8941c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:18.950687 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16412fc3-67f2-4359-aad8-3a02e4c8941c admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:18.951241 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16412fc3-67f2-4359-aad8-3a02e4c8941c admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:18.951366 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16412fc3-67f2-4359-aad8-3a02e4c8941c admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:18.952237 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16412fc3-67f2-4359-aad8-3a02e4c8941c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:18.954124 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16412fc3-67f2-4359-aad8-3a02e4c8941c admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:18.963117 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 918/1840] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:18 2022] PUT /identity/v3/projects/7504520e70d345bf930e89e385957869/users/bf0be741f0cc402e93251339a7ca98ed/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:11:18.965821 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb68c3-1ea0-493f-98b6-93e22a0eafb3 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:18.966112 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb68c3-1ea0-493f-98b6-93e22a0eafb3 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:18.966395 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb68c3-1ea0-493f-98b6-93e22a0eafb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:18.969193 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 923/1841] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:11:18.970241 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16412fc3-67f2-4359-aad8-3a02e4c8941c admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:18.981619 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16412fc3-67f2-4359-aad8-3a02e4c8941c admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:19.019402 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-02bb68c3-1ea0-493f-98b6-93e22a0eafb3 None None] MFA Rules not processed for user `bf0be741f0cc402e93251339a7ca98ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:19.020874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-02bb68c3-1ea0-493f-98b6-93e22a0eafb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:19.055972 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f25ceea-09d0-4065-bbfb-3d3a8b0131e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:19.056748 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25ceea-09d0-4065-bbfb-3d3a8b0131e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:19.057094 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25ceea-09d0-4065-bbfb-3d3a8b0131e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:19.057441 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f25ceea-09d0-4065-bbfb-3d3a8b0131e0 admin admin] PATH_INFO: `/v3/projects/4fdbb347b6fc405e983f5b456f5e0bb2/users/695df8101cb94060a488840999d05fa2/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:19.059713 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f25ceea-09d0-4065-bbfb-3d3a8b0131e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fdbb347b6fc405e983f5b456f5e0bb2, user_id=695df8101cb94060a488840999d05fa2, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:19.062813 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f25ceea-09d0-4065-bbfb-3d3a8b0131e0 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:19.075101 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 924/1842] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:18 2022] PUT /identity/v3/projects/4fdbb347b6fc405e983f5b456f5e0bb2/users/695df8101cb94060a488840999d05fa2/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:19.084169 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:19.087632 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6388412-c203-4c7c-8594-ae52e348eacc None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:19.088047 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6388412-c203-4c7c-8594-ae52e348eacc None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:19.088835 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6388412-c203-4c7c-8594-ae52e348eacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:19.113231 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 919/1843] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:18 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:19.127422 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb68c3-1ea0-493f-98b6-93e22a0eafb3 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:19.129623 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-02bb68c3-1ea0-493f-98b6-93e22a0eafb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:19.129764 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-b6388412-c203-4c7c-8594-ae52e348eacc None None] MFA Rules not processed for user `695df8101cb94060a488840999d05fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:19.130860 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-b6388412-c203-4c7c-8594-ae52e348eacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:19.202109 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb68c3-1ea0-493f-98b6-93e22a0eafb3 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:19.211674 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 925/1844] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:19 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:19.223853 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6388412-c203-4c7c-8594-ae52e348eacc None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:19.228545 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-b6388412-c203-4c7c-8594-ae52e348eacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:19.253852 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:19.276928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4866a-4afb-4949-9b60-42ff5bbef575 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:19.278696 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4866a-4afb-4949-9b60-42ff5bbef575 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:19.278696 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4866a-4afb-4949-9b60-42ff5bbef575 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:19.278696 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4866a-4afb-4949-9b60-42ff5bbef575 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:19.316487 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6388412-c203-4c7c-8594-ae52e348eacc None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:19.320652 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f0b708-ceb4-46f1-a526-dee84de5743b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:19.321264 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0b708-ceb4-46f1-a526-dee84de5743b service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:19.321536 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0b708-ceb4-46f1-a526-dee84de5743b service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:19.321801 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f0b708-ceb4-46f1-a526-dee84de5743b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:19.330763 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f0b708-ceb4-46f1-a526-dee84de5743b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:19.332216 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f0b708-ceb4-46f1-a526-dee84de5743b service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:19.343215 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb4866a-4afb-4949-9b60-42ff5bbef575 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:19.345038 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb4866a-4afb-4949-9b60-42ff5bbef575 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:19.345683 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 926/1845] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:19 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 13 13:11:19.356967 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 920/1846] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:19 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:11:19.540754 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f0b708-ceb4-46f1-a526-dee84de5743b service cinder] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:19.552578 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f0b708-ceb4-46f1-a526-dee84de5743b service cinder] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:19.556641 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8180fd2c-2304-42e2-a68b-9d070bdc5110 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:19.557242 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8180fd2c-2304-42e2-a68b-9d070bdc5110 service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:19.557526 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8180fd2c-2304-42e2-a68b-9d070bdc5110 service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:19.557801 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8180fd2c-2304-42e2-a68b-9d070bdc5110 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:19.616282 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8180fd2c-2304-42e2-a68b-9d070bdc5110 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:19.617752 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8180fd2c-2304-42e2-a68b-9d070bdc5110 service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:19.626705 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 927/1847] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:19 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:11:19.774591 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4866a-4afb-4949-9b60-42ff5bbef575 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:19.828354 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:19.842653 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4866a-4afb-4949-9b60-42ff5bbef575 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:19.847729 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8daad2a-dc4d-404a-8b19-15fa711ed9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:19.848498 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8daad2a-dc4d-404a-8b19-15fa711ed9d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:19.848798 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8daad2a-dc4d-404a-8b19-15fa711ed9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:19.849081 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8daad2a-dc4d-404a-8b19-15fa711ed9d4 admin admin] PATH_INFO: `/v3/users/e552ea32209f4518b7dbe0607ea217f4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:19.850255 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8daad2a-dc4d-404a-8b19-15fa711ed9d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e552ea32209f4518b7dbe0607ea217f4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:19.852225 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8daad2a-dc4d-404a-8b19-15fa711ed9d4 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:19.902491 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c8daad2a-dc4d-404a-8b19-15fa711ed9d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e552ea32209f4518b7dbe0607ea217f4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:19.910499 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c8daad2a-dc4d-404a-8b19-15fa711ed9d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e552ea32209f4518b7dbe0607ea217f4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:19.911323 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c8daad2a-dc4d-404a-8b19-15fa711ed9d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e552ea32209f4518b7dbe0607ea217f4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:19.926045 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c8daad2a-dc4d-404a-8b19-15fa711ed9d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e552ea32209f4518b7dbe0607ea217f4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:19.936708 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c8daad2a-dc4d-404a-8b19-15fa711ed9d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e552ea32209f4518b7dbe0607ea217f4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:19.946777 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 921/1848] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:11:19 2022] DELETE /identity/v3/users/e552ea32209f4518b7dbe0607ea217f4 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:19.963176 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c272e1a-e405-49a2-aa8e-d69205332756 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:19.963176 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c272e1a-e405-49a2-aa8e-d69205332756 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:19.963176 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c272e1a-e405-49a2-aa8e-d69205332756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:19.986049 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-2c272e1a-e405-49a2-aa8e-d69205332756 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:19.987294 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-2c272e1a-e405-49a2-aa8e-d69205332756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:20.058564 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 928/1849] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:20.068590 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8daad2a-dc4d-404a-8b19-15fa711ed9d4 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:20.070057 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c8daad2a-dc4d-404a-8b19-15fa711ed9d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:20.127954 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8daad2a-dc4d-404a-8b19-15fa711ed9d4 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:20.131607 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac64b8-bc4d-4a94-a0a3-abdb2494bf67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:20.132208 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac64b8-bc4d-4a94-a0a3-abdb2494bf67 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:20.132485 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac64b8-bc4d-4a94-a0a3-abdb2494bf67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:20.132754 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac64b8-bc4d-4a94-a0a3-abdb2494bf67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:20.141574 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac64b8-bc4d-4a94-a0a3-abdb2494bf67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:20.142980 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac64b8-bc4d-4a94-a0a3-abdb2494bf67 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:20.155335 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 922/1850] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:11:20.520669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c272e1a-e405-49a2-aa8e-d69205332756 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:20.520669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-2c272e1a-e405-49a2-aa8e-d69205332756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:20.528741 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c272e1a-e405-49a2-aa8e-d69205332756 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:20.533382 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20782604-88f7-45d6-8221-890062380c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:20.534018 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20782604-88f7-45d6-8221-890062380c7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:20.534300 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20782604-88f7-45d6-8221-890062380c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:20.534576 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20782604-88f7-45d6-8221-890062380c7f admin admin] PATH_INFO: `/v3/projects/88b27ae498a74fd0aca979f928cf1742` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:20.535800 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20782604-88f7-45d6-8221-890062380c7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=88b27ae498a74fd0aca979f928cf1742)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:20.538498 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20782604-88f7-45d6-8221-890062380c7f admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:20.586910 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-20782604-88f7-45d6-8221-890062380c7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88b27ae498a74fd0aca979f928cf1742', 'request_id': 'req-20782604-88f7-45d6-8221-890062380c7f'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:20.589234 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:20.594084 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-20782604-88f7-45d6-8221-890062380c7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88b27ae498a74fd0aca979f928cf1742', 'request_id': 'req-20782604-88f7-45d6-8221-890062380c7f'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:20.601095 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-20782604-88f7-45d6-8221-890062380c7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88b27ae498a74fd0aca979f928cf1742', 'request_id': 'req-20782604-88f7-45d6-8221-890062380c7f'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:20.613108 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-20782604-88f7-45d6-8221-890062380c7f admin admin] The token cache is being invalidate because project 88b27ae498a74fd0aca979f928cf1742 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:11:20.613383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-20782604-88f7-45d6-8221-890062380c7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:20.613998 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-20782604-88f7-45d6-8221-890062380c7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:20.615499 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 929/1851] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:11:20 2022] DELETE /identity/v3/projects/88b27ae498a74fd0aca979f928cf1742 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:20.619660 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac64b8-bc4d-4a94-a0a3-abdb2494bf67 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:20.629033 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac64b8-bc4d-4a94-a0a3-abdb2494bf67 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:20.685780 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6090b91-be7e-49b6-ae5c-77548c09bc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:20.686520 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6090b91-be7e-49b6-ae5c-77548c09bc9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:20.686832 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6090b91-be7e-49b6-ae5c-77548c09bc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:20.687149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6090b91-be7e-49b6-ae5c-77548c09bc9e admin admin] PATH_INFO: `/v3/users/695df8101cb94060a488840999d05fa2` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:20.688506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6090b91-be7e-49b6-ae5c-77548c09bc9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=695df8101cb94060a488840999d05fa2)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:20.690439 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6090b91-be7e-49b6-ae5c-77548c09bc9e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:20.704227 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:20.738683 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b6090b91-be7e-49b6-ae5c-77548c09bc9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '695df8101cb94060a488840999d05fa2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:20.745523 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b6090b91-be7e-49b6-ae5c-77548c09bc9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '695df8101cb94060a488840999d05fa2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:20.746186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b6090b91-be7e-49b6-ae5c-77548c09bc9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '695df8101cb94060a488840999d05fa2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:20.752687 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b6090b91-be7e-49b6-ae5c-77548c09bc9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '695df8101cb94060a488840999d05fa2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:20.765188 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b6090b91-be7e-49b6-ae5c-77548c09bc9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '695df8101cb94060a488840999d05fa2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:20.770941 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 923/1852] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:11:20 2022] DELETE /identity/v3/users/695df8101cb94060a488840999d05fa2 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:21.056039 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20782604-88f7-45d6-8221-890062380c7f admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:21.058991 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-20782604-88f7-45d6-8221-890062380c7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:21.087707 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20782604-88f7-45d6-8221-890062380c7f admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:21.167040 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c59a4-1905-4d7b-8b36-e1001883eb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:21.167718 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c59a4-1905-4d7b-8b36-e1001883eb42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:21.168039 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c59a4-1905-4d7b-8b36-e1001883eb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:21.168284 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c59a4-1905-4d7b-8b36-e1001883eb42 admin admin] PATH_INFO: `/v3/projects/4fdbb347b6fc405e983f5b456f5e0bb2` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:21.169414 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40c59a4-1905-4d7b-8b36-e1001883eb42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fdbb347b6fc405e983f5b456f5e0bb2)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:21.171403 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40c59a4-1905-4d7b-8b36-e1001883eb42 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:21.219287 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-a40c59a4-1905-4d7b-8b36-e1001883eb42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fdbb347b6fc405e983f5b456f5e0bb2', 'request_id': 'req-a40c59a4-1905-4d7b-8b36-e1001883eb42'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:21.227786 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-a40c59a4-1905-4d7b-8b36-e1001883eb42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fdbb347b6fc405e983f5b456f5e0bb2', 'request_id': 'req-a40c59a4-1905-4d7b-8b36-e1001883eb42'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:21.235603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-a40c59a4-1905-4d7b-8b36-e1001883eb42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fdbb347b6fc405e983f5b456f5e0bb2', 'request_id': 'req-a40c59a4-1905-4d7b-8b36-e1001883eb42'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:21.250101 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-a40c59a4-1905-4d7b-8b36-e1001883eb42 admin admin] The token cache is being invalidate because project 4fdbb347b6fc405e983f5b456f5e0bb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:11:21.250392 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-a40c59a4-1905-4d7b-8b36-e1001883eb42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:21.251023 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-a40c59a4-1905-4d7b-8b36-e1001883eb42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:21.252682 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 930/1853] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:11:21 2022] DELETE /identity/v3/projects/4fdbb347b6fc405e983f5b456f5e0bb2 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:21.258002 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340968b-2085-4a2b-b28c-4b3947e3aa8f None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:21.258487 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340968b-2085-4a2b-b28c-4b3947e3aa8f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:21.258598 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340968b-2085-4a2b-b28c-4b3947e3aa8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:21.282240 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d340968b-2085-4a2b-b28c-4b3947e3aa8f None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:21.283036 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d340968b-2085-4a2b-b28c-4b3947e3aa8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:21.342493 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 924/1854] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:21.350723 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448d3cd-3512-4378-a3ae-4a1c8bc5f5ef None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:21.351013 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448d3cd-3512-4378-a3ae-4a1c8bc5f5ef None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:21.351329 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448d3cd-3512-4378-a3ae-4a1c8bc5f5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:21.370858 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-c448d3cd-3512-4378-a3ae-4a1c8bc5f5ef None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:21.372118 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-c448d3cd-3512-4378-a3ae-4a1c8bc5f5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:21.380718 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 931/1855] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:11:21.385903 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340968b-2085-4a2b-b28c-4b3947e3aa8f None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:21.395775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340968b-2085-4a2b-b28c-4b3947e3aa8f None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:21.400845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21628717-4d13-4da2-8ca5-f99d3fba2e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:21.401548 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21628717-4d13-4da2-8ca5-f99d3fba2e1d admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:21.401830 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21628717-4d13-4da2-8ca5-f99d3fba2e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:21.402190 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21628717-4d13-4da2-8ca5-f99d3fba2e1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:21.402841 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21628717-4d13-4da2-8ca5-f99d3fba2e1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:11:21.403303 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21628717-4d13-4da2-8ca5-f99d3fba2e1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:21.404906 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21628717-4d13-4da2-8ca5-f99d3fba2e1d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:21.420436 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 925/1856] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:11:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:11:21.424962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448d3cd-3512-4378-a3ae-4a1c8bc5f5ef None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:21.434410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448d3cd-3512-4378-a3ae-4a1c8bc5f5ef None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:21.438904 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1435b-f0e2-4916-be1b-1828ef314441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:21.439552 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1435b-f0e2-4916-be1b-1828ef314441 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:21.439823 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1435b-f0e2-4916-be1b-1828ef314441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:21.440094 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1435b-f0e2-4916-be1b-1828ef314441 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:21.440716 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be1435b-f0e2-4916-be1b-1828ef314441 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:21.442596 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be1435b-f0e2-4916-be1b-1828ef314441 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:21.468737 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 932/1857] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:11:21 2022] POST /identity/v3/projects => generated 363 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:21.472904 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21628717-4d13-4da2-8ca5-f99d3fba2e1d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:21.483680 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21628717-4d13-4da2-8ca5-f99d3fba2e1d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:21.554124 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19cae8-ce67-47e9-9e7e-636863a9780d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:21.554740 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19cae8-ce67-47e9-9e7e-636863a9780d admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:21.555024 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19cae8-ce67-47e9-9e7e-636863a9780d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:21.555340 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19cae8-ce67-47e9-9e7e-636863a9780d admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:21.555997 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c19cae8-ce67-47e9-9e7e-636863a9780d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:21.557957 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c19cae8-ce67-47e9-9e7e-636863a9780d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:21.579329 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 926/1858] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:11:21 2022] POST /identity/v3/users => generated 282 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:11:21.583408 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1435b-f0e2-4916-be1b-1828ef314441 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:21.594446 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1435b-f0e2-4916-be1b-1828ef314441 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:21.599591 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4cacf0-f105-4981-bfc0-ccd15bed6741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:21.600199 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4cacf0-f105-4981-bfc0-ccd15bed6741 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:21.600483 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4cacf0-f105-4981-bfc0-ccd15bed6741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:21.600756 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4cacf0-f105-4981-bfc0-ccd15bed6741 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:21.601414 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4cacf0-f105-4981-bfc0-ccd15bed6741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:21.603042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4cacf0-f105-4981-bfc0-ccd15bed6741 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:21.613939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 933/1859] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:11:21.617844 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19cae8-ce67-47e9-9e7e-636863a9780d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:21.628182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19cae8-ce67-47e9-9e7e-636863a9780d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:21.633726 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e337634b-ed45-417a-9b7b-8bc557ef0d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:21.634475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e337634b-ed45-417a-9b7b-8bc557ef0d47 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:21.634998 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e337634b-ed45-417a-9b7b-8bc557ef0d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:21.635350 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e337634b-ed45-417a-9b7b-8bc557ef0d47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:21.636380 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e337634b-ed45-417a-9b7b-8bc557ef0d47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:21.638613 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e337634b-ed45-417a-9b7b-8bc557ef0d47 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:21.640583 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:21.657268 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 927/1860] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:21.661505 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4cacf0-f105-4981-bfc0-ccd15bed6741 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:21.671695 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4cacf0-f105-4981-bfc0-ccd15bed6741 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:21.677070 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4670f3-1a03-48b0-b44e-50e7f10edb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:21.677729 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4670f3-1a03-48b0-b44e-50e7f10edb25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:21.678030 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4670f3-1a03-48b0-b44e-50e7f10edb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:21.678315 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4670f3-1a03-48b0-b44e-50e7f10edb25 admin admin] PATH_INFO: `/v3/projects/7aa6e33feedf43c0ac2b9bb673302cca/users/96c231a8e7624fc2bfa2821fc1f15389/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:21.697498 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4670f3-1a03-48b0-b44e-50e7f10edb25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aa6e33feedf43c0ac2b9bb673302cca, user_id=96c231a8e7624fc2bfa2821fc1f15389, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:21.700429 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4670f3-1a03-48b0-b44e-50e7f10edb25 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:21.715125 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 934/1861] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:21 2022] PUT /identity/v3/projects/7aa6e33feedf43c0ac2b9bb673302cca/users/96c231a8e7624fc2bfa2821fc1f15389/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:11:21.726626 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02152f-13f9-49f8-b1da-942ea3ff0458 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:21.726911 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02152f-13f9-49f8-b1da-942ea3ff0458 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:21.727223 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02152f-13f9-49f8-b1da-942ea3ff0458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:21.756970 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-fb02152f-13f9-49f8-b1da-942ea3ff0458 None None] MFA Rules not processed for user `96c231a8e7624fc2bfa2821fc1f15389`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:21.758212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fb02152f-13f9-49f8-b1da-942ea3ff0458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:21.822601 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 928/1862] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:21.827711 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4670f3-1a03-48b0-b44e-50e7f10edb25 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:21.837823 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:21.893018 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4670f3-1a03-48b0-b44e-50e7f10edb25 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:21.898151 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444f0e3-2312-465f-b267-169c2a1900ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:21.898748 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444f0e3-2312-465f-b267-169c2a1900ba admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:21.899025 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444f0e3-2312-465f-b267-169c2a1900ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:21.899329 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444f0e3-2312-465f-b267-169c2a1900ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:21.899952 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a444f0e3-2312-465f-b267-169c2a1900ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:21.901874 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a444f0e3-2312-465f-b267-169c2a1900ba admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:21.929002 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 935/1863] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:11:21 2022] POST /identity/v3/projects => generated 363 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:21.933347 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb02152f-13f9-49f8-b1da-942ea3ff0458 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:21.947379 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb02152f-13f9-49f8-b1da-942ea3ff0458 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:22.003810 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a632194-9e15-414c-8740-908f127b14fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:22.004415 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a632194-9e15-414c-8740-908f127b14fb admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:22.004697 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a632194-9e15-414c-8740-908f127b14fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:22.004978 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a632194-9e15-414c-8740-908f127b14fb admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:22.005617 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a632194-9e15-414c-8740-908f127b14fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:22.008097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a632194-9e15-414c-8740-908f127b14fb admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:22.030417 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 929/1864] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:11:21 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:22.032752 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444f0e3-2312-465f-b267-169c2a1900ba admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:22.044317 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444f0e3-2312-465f-b267-169c2a1900ba admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:22.049610 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e711b6-64fc-4624-9cf5-5ee52a938dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:22.050251 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e711b6-64fc-4624-9cf5-5ee52a938dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:22.050533 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e711b6-64fc-4624-9cf5-5ee52a938dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:22.050802 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e711b6-64fc-4624-9cf5-5ee52a938dd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:22.051519 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e711b6-64fc-4624-9cf5-5ee52a938dd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:22.054090 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e711b6-64fc-4624-9cf5-5ee52a938dd5 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:22.066458 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 936/1865] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:22.070684 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a632194-9e15-414c-8740-908f127b14fb admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:22.080594 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a632194-9e15-414c-8740-908f127b14fb admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:22.085044 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac4f64-7111-4416-b4d3-efed40351c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:22.085680 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac4f64-7111-4416-b4d3-efed40351c43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:22.085955 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac4f64-7111-4416-b4d3-efed40351c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:22.086223 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac4f64-7111-4416-b4d3-efed40351c43 admin admin] PATH_INFO: `/v3/projects/b35bc92e4e99480b9e8623be302476c3/users/18361843186546ae9f9002f7ad1f169f/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:22.101333 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ac4f64-7111-4416-b4d3-efed40351c43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b35bc92e4e99480b9e8623be302476c3, user_id=18361843186546ae9f9002f7ad1f169f, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:22.104000 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ac4f64-7111-4416-b4d3-efed40351c43 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:22.116800 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 930/1866] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:22 2022] PUT /identity/v3/projects/b35bc92e4e99480b9e8623be302476c3/users/18361843186546ae9f9002f7ad1f169f/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:22.120775 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e711b6-64fc-4624-9cf5-5ee52a938dd5 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:22.130817 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e711b6-64fc-4624-9cf5-5ee52a938dd5 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:22.194520 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea5a5c-2401-4035-95d1-cb6c94a12637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:22.195161 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea5a5c-2401-4035-95d1-cb6c94a12637 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:22.195441 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea5a5c-2401-4035-95d1-cb6c94a12637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:22.195712 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea5a5c-2401-4035-95d1-cb6c94a12637 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:22.196352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfea5a5c-2401-4035-95d1-cb6c94a12637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:22.197934 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfea5a5c-2401-4035-95d1-cb6c94a12637 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:22.207595 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 937/1867] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:22 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:22.212426 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac4f64-7111-4416-b4d3-efed40351c43 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:22.222448 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac4f64-7111-4416-b4d3-efed40351c43 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:22.227600 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c36273-9559-40da-b451-63cd997273fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:22.228242 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c36273-9559-40da-b451-63cd997273fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:22.228524 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c36273-9559-40da-b451-63cd997273fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:22.228798 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c36273-9559-40da-b451-63cd997273fd admin admin] PATH_INFO: `/v3/projects/b35bc92e4e99480b9e8623be302476c3/users/18361843186546ae9f9002f7ad1f169f/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:22.230678 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c36273-9559-40da-b451-63cd997273fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b35bc92e4e99480b9e8623be302476c3, user_id=18361843186546ae9f9002f7ad1f169f, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:22.233335 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c36273-9559-40da-b451-63cd997273fd admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:22.244651 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 931/1868] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:22 2022] PUT /identity/v3/projects/b35bc92e4e99480b9e8623be302476c3/users/18361843186546ae9f9002f7ad1f169f/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:22.255631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a54ee2-a922-4d7e-86ad-ab8789a2be76 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:22.255947 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a54ee2-a922-4d7e-86ad-ab8789a2be76 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:22.256731 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a54ee2-a922-4d7e-86ad-ab8789a2be76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:22.285365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-b1a54ee2-a922-4d7e-86ad-ab8789a2be76 None None] MFA Rules not processed for user `18361843186546ae9f9002f7ad1f169f`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:22.286602 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-b1a54ee2-a922-4d7e-86ad-ab8789a2be76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:22.352938 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 938/1869] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:22 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:22.364089 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c36273-9559-40da-b451-63cd997273fd admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:22.365899 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-69c36273-9559-40da-b451-63cd997273fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:22.409239 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:22.424628 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c36273-9559-40da-b451-63cd997273fd admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:22.428478 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c8ade-a118-4788-ad84-42313ee8514b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:22.429067 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c8ade-a118-4788-ad84-42313ee8514b service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:22.429342 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c8ade-a118-4788-ad84-42313ee8514b service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:22.429611 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c8ade-a118-4788-ad84-42313ee8514b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:22.438388 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5c8ade-a118-4788-ad84-42313ee8514b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:22.439854 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5c8ade-a118-4788-ad84-42313ee8514b service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:22.450416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 932/1870] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:22 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:11:22.525590 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a54ee2-a922-4d7e-86ad-ab8789a2be76 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:22.530342 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:22.536728 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a54ee2-a922-4d7e-86ad-ab8789a2be76 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:22.541098 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb3543-3d4e-43d6-9057-547c40b072bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:22.541704 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb3543-3d4e-43d6-9057-547c40b072bd service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:22.541984 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb3543-3d4e-43d6-9057-547c40b072bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:22.542258 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb3543-3d4e-43d6-9057-547c40b072bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:22.618385 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fb3543-3d4e-43d6-9057-547c40b072bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:22.618385 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fb3543-3d4e-43d6-9057-547c40b072bd service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:22.626684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 939/1871] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 13 13:11:26.422463 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c8ade-a118-4788-ad84-42313ee8514b service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:26.491458 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c8ade-a118-4788-ad84-42313ee8514b service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:26.495896 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a057c351-c666-4c5c-a27c-2062c8d2b6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:26.496509 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057c351-c666-4c5c-a27c-2062c8d2b6fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:26.496782 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057c351-c666-4c5c-a27c-2062c8d2b6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:26.497051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057c351-c666-4c5c-a27c-2062c8d2b6fd admin admin] PATH_INFO: `/v3/users/96c231a8e7624fc2bfa2821fc1f15389` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:26.498138 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a057c351-c666-4c5c-a27c-2062c8d2b6fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=96c231a8e7624fc2bfa2821fc1f15389)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:26.500073 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a057c351-c666-4c5c-a27c-2062c8d2b6fd admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:26.545116 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a057c351-c666-4c5c-a27c-2062c8d2b6fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96c231a8e7624fc2bfa2821fc1f15389'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:26.552164 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a057c351-c666-4c5c-a27c-2062c8d2b6fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96c231a8e7624fc2bfa2821fc1f15389'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:26.552902 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a057c351-c666-4c5c-a27c-2062c8d2b6fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96c231a8e7624fc2bfa2821fc1f15389'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:26.560063 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a057c351-c666-4c5c-a27c-2062c8d2b6fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96c231a8e7624fc2bfa2821fc1f15389'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:26.577867 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a057c351-c666-4c5c-a27c-2062c8d2b6fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96c231a8e7624fc2bfa2821fc1f15389'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:26.583381 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 933/1872] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:11:26 2022] DELETE /identity/v3/users/96c231a8e7624fc2bfa2821fc1f15389 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:26.589447 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a283b-ab12-4f7e-89a1-c021bb08ea37 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:26.589923 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a283b-ab12-4f7e-89a1-c021bb08ea37 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:26.590012 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a283b-ab12-4f7e-89a1-c021bb08ea37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:26.609565 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-199a283b-ab12-4f7e-89a1-c021bb08ea37 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:26.610709 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-199a283b-ab12-4f7e-89a1-c021bb08ea37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:26.667531 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 940/1873] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:26.677492 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a057c351-c666-4c5c-a27c-2062c8d2b6fd admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:26.679068 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a057c351-c666-4c5c-a27c-2062c8d2b6fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:26.740635 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a057c351-c666-4c5c-a27c-2062c8d2b6fd admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:26.744215 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e8a99-5152-4ff2-9a82-b151dd2aff34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:26.744843 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e8a99-5152-4ff2-9a82-b151dd2aff34 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:26.745119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e8a99-5152-4ff2-9a82-b151dd2aff34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:26.745387 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e8a99-5152-4ff2-9a82-b151dd2aff34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:26.754052 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23e8a99-5152-4ff2-9a82-b151dd2aff34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:26.755496 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23e8a99-5152-4ff2-9a82-b151dd2aff34 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:26.765115 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 934/1874] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:11:27.074125 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a283b-ab12-4f7e-89a1-c021bb08ea37 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:27.075956 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-199a283b-ab12-4f7e-89a1-c021bb08ea37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:27.089664 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a283b-ab12-4f7e-89a1-c021bb08ea37 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:27.094701 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936adc5-40cf-4063-9d17-e53952b556fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:27.095463 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936adc5-40cf-4063-9d17-e53952b556fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:27.095802 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936adc5-40cf-4063-9d17-e53952b556fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:27.096145 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936adc5-40cf-4063-9d17-e53952b556fd admin admin] PATH_INFO: `/v3/projects/7aa6e33feedf43c0ac2b9bb673302cca` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:27.097437 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936adc5-40cf-4063-9d17-e53952b556fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7aa6e33feedf43c0ac2b9bb673302cca)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:27.099398 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936adc5-40cf-4063-9d17-e53952b556fd admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:27.148012 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c936adc5-40cf-4063-9d17-e53952b556fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7aa6e33feedf43c0ac2b9bb673302cca', 'request_id': 'req-c936adc5-40cf-4063-9d17-e53952b556fd'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:27.154840 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c936adc5-40cf-4063-9d17-e53952b556fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7aa6e33feedf43c0ac2b9bb673302cca', 'request_id': 'req-c936adc5-40cf-4063-9d17-e53952b556fd'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:27.162230 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c936adc5-40cf-4063-9d17-e53952b556fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7aa6e33feedf43c0ac2b9bb673302cca', 'request_id': 'req-c936adc5-40cf-4063-9d17-e53952b556fd'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:27.174912 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c936adc5-40cf-4063-9d17-e53952b556fd admin admin] The token cache is being invalidate because project 7aa6e33feedf43c0ac2b9bb673302cca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:11:27.175246 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c936adc5-40cf-4063-9d17-e53952b556fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:27.175862 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c936adc5-40cf-4063-9d17-e53952b556fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:27.177616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 941/1875] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:11:27 2022] DELETE /identity/v3/projects/7aa6e33feedf43c0ac2b9bb673302cca => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:27.182110 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e8a99-5152-4ff2-9a82-b151dd2aff34 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:27.187132 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:27.194532 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e8a99-5152-4ff2-9a82-b151dd2aff34 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:27.252509 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f621b9-da0f-44b8-a007-7d44d2c8ce5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:27.253145 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f621b9-da0f-44b8-a007-7d44d2c8ce5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:27.253429 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f621b9-da0f-44b8-a007-7d44d2c8ce5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:27.253702 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f621b9-da0f-44b8-a007-7d44d2c8ce5a admin admin] PATH_INFO: `/v3/users/18361843186546ae9f9002f7ad1f169f` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:27.254902 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f621b9-da0f-44b8-a007-7d44d2c8ce5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=18361843186546ae9f9002f7ad1f169f)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:27.257448 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f621b9-da0f-44b8-a007-7d44d2c8ce5a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:27.306587 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-29f621b9-da0f-44b8-a007-7d44d2c8ce5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18361843186546ae9f9002f7ad1f169f'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:27.313307 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-29f621b9-da0f-44b8-a007-7d44d2c8ce5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18361843186546ae9f9002f7ad1f169f'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:27.313965 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-29f621b9-da0f-44b8-a007-7d44d2c8ce5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18361843186546ae9f9002f7ad1f169f'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:27.321643 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-29f621b9-da0f-44b8-a007-7d44d2c8ce5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18361843186546ae9f9002f7ad1f169f'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:27.333240 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-29f621b9-da0f-44b8-a007-7d44d2c8ce5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18361843186546ae9f9002f7ad1f169f'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:27.338963 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 935/1876] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:11:27 2022] DELETE /identity/v3/users/18361843186546ae9f9002f7ad1f169f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:27.703141 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936adc5-40cf-4063-9d17-e53952b556fd admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:27.704640 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-c936adc5-40cf-4063-9d17-e53952b556fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:27.720367 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936adc5-40cf-4063-9d17-e53952b556fd admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:27.741797 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:27.780132 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abafbdc2-19dd-4cac-a405-4083204dca11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:27.780785 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abafbdc2-19dd-4cac-a405-4083204dca11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:27.781090 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abafbdc2-19dd-4cac-a405-4083204dca11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:27.781364 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abafbdc2-19dd-4cac-a405-4083204dca11 admin admin] PATH_INFO: `/v3/projects/b35bc92e4e99480b9e8623be302476c3` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:27.782533 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abafbdc2-19dd-4cac-a405-4083204dca11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b35bc92e4e99480b9e8623be302476c3)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:27.784536 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abafbdc2-19dd-4cac-a405-4083204dca11 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:27.830721 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-abafbdc2-19dd-4cac-a405-4083204dca11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b35bc92e4e99480b9e8623be302476c3', 'request_id': 'req-abafbdc2-19dd-4cac-a405-4083204dca11'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:27.836196 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-abafbdc2-19dd-4cac-a405-4083204dca11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b35bc92e4e99480b9e8623be302476c3', 'request_id': 'req-abafbdc2-19dd-4cac-a405-4083204dca11'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:27.847427 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-abafbdc2-19dd-4cac-a405-4083204dca11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b35bc92e4e99480b9e8623be302476c3', 'request_id': 'req-abafbdc2-19dd-4cac-a405-4083204dca11'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:27.863228 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-abafbdc2-19dd-4cac-a405-4083204dca11 admin admin] The token cache is being invalidate because project b35bc92e4e99480b9e8623be302476c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:11:27.863524 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-abafbdc2-19dd-4cac-a405-4083204dca11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:27.864162 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-abafbdc2-19dd-4cac-a405-4083204dca11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:27.865729 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 942/1877] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:11:27 2022] DELETE /identity/v3/projects/b35bc92e4e99480b9e8623be302476c3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:27.871861 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92919b4e-bbe9-4aad-856a-88abc01059cc None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:27.872312 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92919b4e-bbe9-4aad-856a-88abc01059cc None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:27.872539 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92919b4e-bbe9-4aad-856a-88abc01059cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:27.892833 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-92919b4e-bbe9-4aad-856a-88abc01059cc None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:27.894283 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-92919b4e-bbe9-4aad-856a-88abc01059cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:27.955435 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 936/1878] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:11:27.963413 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f4679-c661-478c-a8db-6c6b648f3820 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:27.963705 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f4679-c661-478c-a8db-6c6b648f3820 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:27.963985 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f4679-c661-478c-a8db-6c6b648f3820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:27.984009 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-ae8f4679-c661-478c-a8db-6c6b648f3820 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:27.985215 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-ae8f4679-c661-478c-a8db-6c6b648f3820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:27.993906 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 943/1879] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:27.999765 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92919b4e-bbe9-4aad-856a-88abc01059cc None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:28.012210 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92919b4e-bbe9-4aad-856a-88abc01059cc None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:28.017323 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce290d2a-4687-405e-bc0e-2778ce392184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:28.017933 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce290d2a-4687-405e-bc0e-2778ce392184 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:28.018210 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce290d2a-4687-405e-bc0e-2778ce392184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:28.018477 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce290d2a-4687-405e-bc0e-2778ce392184 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:28.019174 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce290d2a-4687-405e-bc0e-2778ce392184 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:11:28.019475 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce290d2a-4687-405e-bc0e-2778ce392184 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:28.022396 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce290d2a-4687-405e-bc0e-2778ce392184 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:28.038854 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 937/1880] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:11:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:11:28.043228 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f4679-c661-478c-a8db-6c6b648f3820 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:28.052928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f4679-c661-478c-a8db-6c6b648f3820 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:28.058121 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04adae-6b1b-47c1-98d4-c805a814c07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:28.058788 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04adae-6b1b-47c1-98d4-c805a814c07c admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:28.059139 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04adae-6b1b-47c1-98d4-c805a814c07c admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:28.059481 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04adae-6b1b-47c1-98d4-c805a814c07c admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:28.060056 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04adae-6b1b-47c1-98d4-c805a814c07c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:28.061950 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04adae-6b1b-47c1-98d4-c805a814c07c admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:28.088153 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 944/1881] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:11:28 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:28.092242 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce290d2a-4687-405e-bc0e-2778ce392184 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:28.102825 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce290d2a-4687-405e-bc0e-2778ce392184 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:28.107438 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:28.111242 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04adae-6b1b-47c1-98d4-c805a814c07c admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:28.112930 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-ca04adae-6b1b-47c1-98d4-c805a814c07c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:28.176035 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b70eab1-6e92-4696-bd74-de9971a847da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:28.176917 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70eab1-6e92-4696-bd74-de9971a847da admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:28.177197 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70eab1-6e92-4696-bd74-de9971a847da admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:28.177470 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70eab1-6e92-4696-bd74-de9971a847da admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:28.178134 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b70eab1-6e92-4696-bd74-de9971a847da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:28.180108 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b70eab1-6e92-4696-bd74-de9971a847da admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:28.182440 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04adae-6b1b-47c1-98d4-c805a814c07c admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:28.187086 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a81fe-a622-438b-9313-e39f893bf936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:28.187732 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a81fe-a622-438b-9313-e39f893bf936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:28.188023 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a81fe-a622-438b-9313-e39f893bf936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:28.188294 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a81fe-a622-438b-9313-e39f893bf936 admin admin] PATH_INFO: `/v3/users/40be4afcbd5e49fa85f596cce44026cd` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:28.189427 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2a81fe-a622-438b-9313-e39f893bf936 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40be4afcbd5e49fa85f596cce44026cd)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:28.191371 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2a81fe-a622-438b-9313-e39f893bf936 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:28.205100 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 938/1882] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:11:28 2022] POST /identity/v3/users => generated 285 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:28.206148 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b70eab1-6e92-4696-bd74-de9971a847da admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:28.216352 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b70eab1-6e92-4696-bd74-de9971a847da admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:28.221576 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30629c-688f-4e75-b987-b8d0794cab3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:28.222175 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30629c-688f-4e75-b987-b8d0794cab3a admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:28.222462 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30629c-688f-4e75-b987-b8d0794cab3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:28.222719 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30629c-688f-4e75-b987-b8d0794cab3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:28.222839 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:28.223394 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea30629c-688f-4e75-b987-b8d0794cab3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:28.225932 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea30629c-688f-4e75-b987-b8d0794cab3a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:28.238230 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 939/1883] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:28.243569 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30629c-688f-4e75-b987-b8d0794cab3a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:28.245055 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-0c2a81fe-a622-438b-9313-e39f893bf936 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40be4afcbd5e49fa85f596cce44026cd'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:28.252783 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-0c2a81fe-a622-438b-9313-e39f893bf936 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40be4afcbd5e49fa85f596cce44026cd'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:28.253477 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-0c2a81fe-a622-438b-9313-e39f893bf936 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40be4afcbd5e49fa85f596cce44026cd'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:28.253608 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30629c-688f-4e75-b987-b8d0794cab3a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:28.254133 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-0c2a81fe-a622-438b-9313-e39f893bf936 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40be4afcbd5e49fa85f596cce44026cd'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:28.260333 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c5fc5-d4e1-485e-8123-6926e83b3c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:28.261082 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c5fc5-d4e1-485e-8123-6926e83b3c97 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:28.261338 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c5fc5-d4e1-485e-8123-6926e83b3c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:28.261628 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c5fc5-d4e1-485e-8123-6926e83b3c97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:28.262307 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476c5fc5-d4e1-485e-8123-6926e83b3c97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:28.262826 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-0c2a81fe-a622-438b-9313-e39f893bf936 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40be4afcbd5e49fa85f596cce44026cd'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:28.264110 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476c5fc5-d4e1-485e-8123-6926e83b3c97 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:28.276335 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 940/1884] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:11:28.280286 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c5fc5-d4e1-485e-8123-6926e83b3c97 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:28.282013 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-476c5fc5-d4e1-485e-8123-6926e83b3c97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:28.286612 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 945/1885] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:11:28 2022] DELETE /identity/v3/users/40be4afcbd5e49fa85f596cce44026cd => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:28.290782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d17927-7775-4799-a333-cfd85243d005 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:28.291076 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d17927-7775-4799-a333-cfd85243d005 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:28.291379 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d17927-7775-4799-a333-cfd85243d005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:28.294708 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c5fc5-d4e1-485e-8123-6926e83b3c97 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:28.310051 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-c0d17927-7775-4799-a333-cfd85243d005 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:28.311270 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-c0d17927-7775-4799-a333-cfd85243d005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:28.354530 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5742a61-8813-40ed-ade3-e5a22b91c5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:28.355234 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5742a61-8813-40ed-ade3-e5a22b91c5e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:28.355581 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5742a61-8813-40ed-ade3-e5a22b91c5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:28.355916 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5742a61-8813-40ed-ade3-e5a22b91c5e8 admin admin] PATH_INFO: `/v3/projects/1a2a63ae7c094355a8b52dcec4a2239c/users/b0abac9ba591401aacd7326f4b50ce38/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:28.369964 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5742a61-8813-40ed-ade3-e5a22b91c5e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a2a63ae7c094355a8b52dcec4a2239c, user_id=b0abac9ba591401aacd7326f4b50ce38, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:28.370233 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 946/1886] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:11:28.372728 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5742a61-8813-40ed-ade3-e5a22b91c5e8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:28.380187 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d17927-7775-4799-a333-cfd85243d005 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:28.382116 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-c0d17927-7775-4799-a333-cfd85243d005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:28.386277 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 941/1887] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:28 2022] PUT /identity/v3/projects/1a2a63ae7c094355a8b52dcec4a2239c/users/b0abac9ba591401aacd7326f4b50ce38/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:11:28.397812 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96115ea2-3688-4276-9c94-1b518684e42f None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:28.398094 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96115ea2-3688-4276-9c94-1b518684e42f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:28.398365 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96115ea2-3688-4276-9c94-1b518684e42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:28.424747 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-96115ea2-3688-4276-9c94-1b518684e42f None None] MFA Rules not processed for user `b0abac9ba591401aacd7326f4b50ce38`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:28.425937 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-96115ea2-3688-4276-9c94-1b518684e42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:28.437535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d17927-7775-4799-a333-cfd85243d005 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:28.482912 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:28.496752 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8064e0a-1c6c-4570-8de8-b5fd7b88d714 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:28.498316 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8064e0a-1c6c-4570-8de8-b5fd7b88d714 service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:28.498784 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8064e0a-1c6c-4570-8de8-b5fd7b88d714 service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:28.499860 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8064e0a-1c6c-4570-8de8-b5fd7b88d714 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:28.502625 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 942/1888] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:11:28.507790 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96115ea2-3688-4276-9c94-1b518684e42f None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:28.518692 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:28.573774 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8064e0a-1c6c-4570-8de8-b5fd7b88d714 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:28.574538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8064e0a-1c6c-4570-8de8-b5fd7b88d714 service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:28.578778 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96115ea2-3688-4276-9c94-1b518684e42f None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:28.583418 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9049383d-d3a0-42e3-9eb6-96d3b1bdd41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:28.584603 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049383d-d3a0-42e3-9eb6-96d3b1bdd41f admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:28.584992 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049383d-d3a0-42e3-9eb6-96d3b1bdd41f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:28.585266 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049383d-d3a0-42e3-9eb6-96d3b1bdd41f admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:28.585526 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 947/1889] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:11:28.585921 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9049383d-d3a0-42e3-9eb6-96d3b1bdd41f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:28.587840 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9049383d-d3a0-42e3-9eb6-96d3b1bdd41f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:28.614499 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 943/1890] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:11:28 2022] POST /identity/v3/projects => generated 367 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:11:28.619268 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8064e0a-1c6c-4570-8de8-b5fd7b88d714 service neutron] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:28.629429 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8064e0a-1c6c-4570-8de8-b5fd7b88d714 service neutron] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:28.702543 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c82ce-fa21-4796-91c7-2c87d07ba544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:28.703196 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c82ce-fa21-4796-91c7-2c87d07ba544 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:28.703730 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c82ce-fa21-4796-91c7-2c87d07ba544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:28.703730 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c82ce-fa21-4796-91c7-2c87d07ba544 admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:28.704367 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c82ce-fa21-4796-91c7-2c87d07ba544 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:28.706317 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c82ce-fa21-4796-91c7-2c87d07ba544 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:28.730274 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 948/1891] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:11:28 2022] POST /identity/v3/users => generated 290 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:28.734595 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9049383d-d3a0-42e3-9eb6-96d3b1bdd41f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:28.746323 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9049383d-d3a0-42e3-9eb6-96d3b1bdd41f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:28.751901 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167491ed-bb01-485e-b8fa-425b83238d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:28.752542 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167491ed-bb01-485e-b8fa-425b83238d5a admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:28.752889 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167491ed-bb01-485e-b8fa-425b83238d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:28.753304 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167491ed-bb01-485e-b8fa-425b83238d5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:28.753976 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167491ed-bb01-485e-b8fa-425b83238d5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:28.756452 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167491ed-bb01-485e-b8fa-425b83238d5a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:28.773726 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 944/1892] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:28.774932 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c82ce-fa21-4796-91c7-2c87d07ba544 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:28.787129 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c82ce-fa21-4796-91c7-2c87d07ba544 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:28.798683 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1706fa2a-50df-4144-bd3a-08e1b70f0d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:28.799972 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706fa2a-50df-4144-bd3a-08e1b70f0d07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:28.800258 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706fa2a-50df-4144-bd3a-08e1b70f0d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:28.800537 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706fa2a-50df-4144-bd3a-08e1b70f0d07 admin admin] PATH_INFO: `/v3/projects/e7657c3cfbec43248732f0881c46e33c/users/9c19a42c75514b93a6363928a9a8914d/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:28.819943 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1706fa2a-50df-4144-bd3a-08e1b70f0d07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7657c3cfbec43248732f0881c46e33c, user_id=9c19a42c75514b93a6363928a9a8914d, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:28.823138 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1706fa2a-50df-4144-bd3a-08e1b70f0d07 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:28.839014 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 949/1893] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:28 2022] PUT /identity/v3/projects/e7657c3cfbec43248732f0881c46e33c/users/9c19a42c75514b93a6363928a9a8914d/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:28.843091 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167491ed-bb01-485e-b8fa-425b83238d5a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:28.856095 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167491ed-bb01-485e-b8fa-425b83238d5a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:28.895136 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1706fa2a-50df-4144-bd3a-08e1b70f0d07 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:28.897168 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-1706fa2a-50df-4144-bd3a-08e1b70f0d07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:28.957121 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f751056-88cc-4a0c-adfe-e335a1638d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:28.958031 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f751056-88cc-4a0c-adfe-e335a1638d58 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:28.958379 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f751056-88cc-4a0c-adfe-e335a1638d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:28.958716 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f751056-88cc-4a0c-adfe-e335a1638d58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:28.959442 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f751056-88cc-4a0c-adfe-e335a1638d58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:28.961278 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f751056-88cc-4a0c-adfe-e335a1638d58 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:28.975085 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 945/1894] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:28.978124 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f751056-88cc-4a0c-adfe-e335a1638d58 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:28.985737 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1706fa2a-50df-4144-bd3a-08e1b70f0d07 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:28.988180 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f751056-88cc-4a0c-adfe-e335a1638d58 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:28.991293 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41620f5b-c83a-4b70-8769-87c5c59e9e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:28.992174 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41620f5b-c83a-4b70-8769-87c5c59e9e2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:28.992451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41620f5b-c83a-4b70-8769-87c5c59e9e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:28.992929 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41620f5b-c83a-4b70-8769-87c5c59e9e2a admin admin] PATH_INFO: `/v3/projects/f92874b7f58c479db71250dc19c31251` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:28.993252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1bc2c9-3dde-4416-9e68-394d574c03d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:28.993887 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1bc2c9-3dde-4416-9e68-394d574c03d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:28.994164 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1bc2c9-3dde-4416-9e68-394d574c03d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:28.994246 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41620f5b-c83a-4b70-8769-87c5c59e9e2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f92874b7f58c479db71250dc19c31251)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:28.994434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1bc2c9-3dde-4416-9e68-394d574c03d2 admin admin] PATH_INFO: `/v3/projects/e7657c3cfbec43248732f0881c46e33c/users/9c19a42c75514b93a6363928a9a8914d/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:28.996666 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41620f5b-c83a-4b70-8769-87c5c59e9e2a admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:28.997490 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1bc2c9-3dde-4416-9e68-394d574c03d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7657c3cfbec43248732f0881c46e33c, user_id=9c19a42c75514b93a6363928a9a8914d, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:29.000226 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1bc2c9-3dde-4416-9e68-394d574c03d2 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:29.013188 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 946/1895] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:28 2022] PUT /identity/v3/projects/e7657c3cfbec43248732f0881c46e33c/users/9c19a42c75514b93a6363928a9a8914d/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:11:29.019921 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:29.024553 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d3c8b-d856-4196-9e74-f271ec7ceef2 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:29.024838 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d3c8b-d856-4196-9e74-f271ec7ceef2 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:29.025116 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d3c8b-d856-4196-9e74-f271ec7ceef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:29.047115 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-41620f5b-c83a-4b70-8769-87c5c59e9e2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f92874b7f58c479db71250dc19c31251', 'request_id': 'req-41620f5b-c83a-4b70-8769-87c5c59e9e2a'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:29.053958 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-5f6d3c8b-d856-4196-9e74-f271ec7ceef2 None None] MFA Rules not processed for user `9c19a42c75514b93a6363928a9a8914d`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:29.054928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-41620f5b-c83a-4b70-8769-87c5c59e9e2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f92874b7f58c479db71250dc19c31251', 'request_id': 'req-41620f5b-c83a-4b70-8769-87c5c59e9e2a'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:29.055351 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5f6d3c8b-d856-4196-9e74-f271ec7ceef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:29.061829 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-41620f5b-c83a-4b70-8769-87c5c59e9e2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f92874b7f58c479db71250dc19c31251', 'request_id': 'req-41620f5b-c83a-4b70-8769-87c5c59e9e2a'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:29.074834 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-41620f5b-c83a-4b70-8769-87c5c59e9e2a admin admin] The token cache is being invalidate because project f92874b7f58c479db71250dc19c31251 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:11:29.075200 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-41620f5b-c83a-4b70-8769-87c5c59e9e2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:29.076198 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-41620f5b-c83a-4b70-8769-87c5c59e9e2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:29.077695 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 950/1896] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:11:28 2022] DELETE /identity/v3/projects/f92874b7f58c479db71250dc19c31251 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:29.081953 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41620f5b-c83a-4b70-8769-87c5c59e9e2a admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:29.083365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-41620f5b-c83a-4b70-8769-87c5c59e9e2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:29.093684 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:29.144670 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 947/1897] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:29 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:11:29.155515 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d3c8b-d856-4196-9e74-f271ec7ceef2 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:29.156951 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5f6d3c8b-d856-4196-9e74-f271ec7ceef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:29.161124 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41620f5b-c83a-4b70-8769-87c5c59e9e2a admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:29.166602 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda0575-0cb3-4286-bda7-bddcb5687670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:29.167344 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda0575-0cb3-4286-bda7-bddcb5687670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:29.167694 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda0575-0cb3-4286-bda7-bddcb5687670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:29.168036 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda0575-0cb3-4286-bda7-bddcb5687670 admin admin] PATH_INFO: `/v3/users/bf0be741f0cc402e93251339a7ca98ed` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:29.169230 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afda0575-0cb3-4286-bda7-bddcb5687670 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf0be741f0cc402e93251339a7ca98ed)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:29.171396 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afda0575-0cb3-4286-bda7-bddcb5687670 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:29.227247 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d3c8b-d856-4196-9e74-f271ec7ceef2 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:29.231086 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921ccc4-f204-4895-b6a5-faa921fda4e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:29.231727 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921ccc4-f204-4895-b6a5-faa921fda4e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:29.232014 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921ccc4-f204-4895-b6a5-faa921fda4e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:29.232297 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921ccc4-f204-4895-b6a5-faa921fda4e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:29.234443 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-afda0575-0cb3-4286-bda7-bddcb5687670 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf0be741f0cc402e93251339a7ca98ed'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:29.242416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-afda0575-0cb3-4286-bda7-bddcb5687670 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf0be741f0cc402e93251339a7ca98ed'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:29.242567 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921ccc4-f204-4895-b6a5-faa921fda4e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:29.243149 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-afda0575-0cb3-4286-bda7-bddcb5687670 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf0be741f0cc402e93251339a7ca98ed'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:29.243852 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-afda0575-0cb3-4286-bda7-bddcb5687670 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf0be741f0cc402e93251339a7ca98ed'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:29.243979 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921ccc4-f204-4895-b6a5-faa921fda4e2 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:29.252288 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-afda0575-0cb3-4286-bda7-bddcb5687670 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf0be741f0cc402e93251339a7ca98ed'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:29.254790 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 948/1898] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:29 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 13 13:11:29.270324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 951/1899] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:11:29 2022] DELETE /identity/v3/users/bf0be741f0cc402e93251339a7ca98ed => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:11:29.560844 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921ccc4-f204-4895-b6a5-faa921fda4e2 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:29.582492 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921ccc4-f204-4895-b6a5-faa921fda4e2 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:29.650884 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015d9f02-5c28-40bf-92c4-214cdf3659cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:29.651556 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d9f02-5c28-40bf-92c4-214cdf3659cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:29.651886 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d9f02-5c28-40bf-92c4-214cdf3659cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:29.652169 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d9f02-5c28-40bf-92c4-214cdf3659cc admin admin] PATH_INFO: `/v3/projects/7504520e70d345bf930e89e385957869` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:29.653261 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015d9f02-5c28-40bf-92c4-214cdf3659cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7504520e70d345bf930e89e385957869)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:29.655220 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015d9f02-5c28-40bf-92c4-214cdf3659cc admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:29.706156 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-015d9f02-5c28-40bf-92c4-214cdf3659cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7504520e70d345bf930e89e385957869', 'request_id': 'req-015d9f02-5c28-40bf-92c4-214cdf3659cc'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:29.712289 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-015d9f02-5c28-40bf-92c4-214cdf3659cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7504520e70d345bf930e89e385957869', 'request_id': 'req-015d9f02-5c28-40bf-92c4-214cdf3659cc'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:29.724967 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-015d9f02-5c28-40bf-92c4-214cdf3659cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7504520e70d345bf930e89e385957869', 'request_id': 'req-015d9f02-5c28-40bf-92c4-214cdf3659cc'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:29.738846 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-015d9f02-5c28-40bf-92c4-214cdf3659cc admin admin] The token cache is being invalidate because project 7504520e70d345bf930e89e385957869 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:11:29.739174 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-015d9f02-5c28-40bf-92c4-214cdf3659cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:29.739838 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-015d9f02-5c28-40bf-92c4-214cdf3659cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:29.742072 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 949/1900] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:11:29 2022] DELETE /identity/v3/projects/7504520e70d345bf930e89e385957869 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:29.746688 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88a749-b580-422c-bac0-6997ce289b9e None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:29.747197 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88a749-b580-422c-bac0-6997ce289b9e None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:29.747300 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88a749-b580-422c-bac0-6997ce289b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:29.775636 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-5b88a749-b580-422c-bac0-6997ce289b9e None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:29.776876 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-5b88a749-b580-422c-bac0-6997ce289b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:29.804586 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:29.850452 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 952/1901] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:29.859882 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfce96-6c52-4ef9-beb0-50fce93fa1ef None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:29.860170 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfce96-6c52-4ef9-beb0-50fce93fa1ef None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:29.860444 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfce96-6c52-4ef9-beb0-50fce93fa1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:29.870139 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:29.882538 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c1bfce96-6c52-4ef9-beb0-50fce93fa1ef None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:29.883847 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c1bfce96-6c52-4ef9-beb0-50fce93fa1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:29.891957 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 950/1902] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:29.896122 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88a749-b580-422c-bac0-6997ce289b9e None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:29.906029 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88a749-b580-422c-bac0-6997ce289b9e None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:29.910601 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1eaf68-6795-4e42-b0fa-8aabbddf2474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:29.911193 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1eaf68-6795-4e42-b0fa-8aabbddf2474 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:29.911474 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1eaf68-6795-4e42-b0fa-8aabbddf2474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:29.911722 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1eaf68-6795-4e42-b0fa-8aabbddf2474 admin admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:29.912996 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1eaf68-6795-4e42-b0fa-8aabbddf2474 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:11:29.913308 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1eaf68-6795-4e42-b0fa-8aabbddf2474 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:29.915175 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1eaf68-6795-4e42-b0fa-8aabbddf2474 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:29.931386 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 953/1903] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:11:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 13:11:29.933681 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfce96-6c52-4ef9-beb0-50fce93fa1ef None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:29.944962 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfce96-6c52-4ef9-beb0-50fce93fa1ef None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:29.952724 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d03d1e3-6537-4b4a-bfcd-db267691dd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:29.953488 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03d1e3-6537-4b4a-bfcd-db267691dd3a admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:29.953869 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03d1e3-6537-4b4a-bfcd-db267691dd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:29.954248 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03d1e3-6537-4b4a-bfcd-db267691dd3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:29.954989 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d03d1e3-6537-4b4a-bfcd-db267691dd3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:29.957678 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d03d1e3-6537-4b4a-bfcd-db267691dd3a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:29.997864 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 951/1904] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:11:29 2022] POST /identity/v3/projects => generated 381 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:30.001927 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1eaf68-6795-4e42-b0fa-8aabbddf2474 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:30.015932 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1eaf68-6795-4e42-b0fa-8aabbddf2474 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:30.100455 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee094ad-4ce8-4901-b315-152d9357a90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:30.101100 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee094ad-4ce8-4901-b315-152d9357a90b admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:30.101727 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee094ad-4ce8-4901-b315-152d9357a90b admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:30.102245 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee094ad-4ce8-4901-b315-152d9357a90b admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:30.103365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee094ad-4ce8-4901-b315-152d9357a90b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:30.105510 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee094ad-4ce8-4901-b315-152d9357a90b admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:30.130865 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 954/1905] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:11:30 2022] POST /identity/v3/users => generated 291 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:30.134999 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d03d1e3-6537-4b4a-bfcd-db267691dd3a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:30.156779 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d03d1e3-6537-4b4a-bfcd-db267691dd3a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:30.163485 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc214af-467f-400e-b51a-c8a0729c6bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:30.165635 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc214af-467f-400e-b51a-c8a0729c6bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:30.166042 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc214af-467f-400e-b51a-c8a0729c6bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:30.166413 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc214af-467f-400e-b51a-c8a0729c6bb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:30.167248 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc214af-467f-400e-b51a-c8a0729c6bb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:30.168989 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc214af-467f-400e-b51a-c8a0729c6bb4 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:30.180323 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 952/1906] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:30 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:30.184237 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee094ad-4ce8-4901-b315-152d9357a90b admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:30.196368 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee094ad-4ce8-4901-b315-152d9357a90b admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:30.201573 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32656b4-5e1c-4132-9264-61c5e8869404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:30.202177 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32656b4-5e1c-4132-9264-61c5e8869404 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:30.202560 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32656b4-5e1c-4132-9264-61c5e8869404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:30.202769 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32656b4-5e1c-4132-9264-61c5e8869404 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:30.203393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32656b4-5e1c-4132-9264-61c5e8869404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:30.205056 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32656b4-5e1c-4132-9264-61c5e8869404 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:30.221416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 955/1907] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:30.222703 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc214af-467f-400e-b51a-c8a0729c6bb4 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:30.233629 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc214af-467f-400e-b51a-c8a0729c6bb4 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:30.238170 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2a51b-0523-4a26-a6f8-10a788e7b8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:30.238820 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2a51b-0523-4a26-a6f8-10a788e7b8ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:30.239184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2a51b-0523-4a26-a6f8-10a788e7b8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:30.239456 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2a51b-0523-4a26-a6f8-10a788e7b8ad admin admin] PATH_INFO: `/v3/projects/03b0bc604e8e474ba837ed92efb7506c/users/641dcf98932a45fe9acad762d72689cd/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:30.255895 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc2a51b-0523-4a26-a6f8-10a788e7b8ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=03b0bc604e8e474ba837ed92efb7506c, user_id=641dcf98932a45fe9acad762d72689cd, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:30.258595 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc2a51b-0523-4a26-a6f8-10a788e7b8ad admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:30.272033 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 953/1908] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:30 2022] PUT /identity/v3/projects/03b0bc604e8e474ba837ed92efb7506c/users/641dcf98932a45fe9acad762d72689cd/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:30.282948 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ce111-d947-485f-a223-bd0ddf6d6865 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:30.283257 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ce111-d947-485f-a223-bd0ddf6d6865 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:30.283538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ce111-d947-485f-a223-bd0ddf6d6865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:30.316218 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-ae6ce111-d947-485f-a223-bd0ddf6d6865 None None] MFA Rules not processed for user `641dcf98932a45fe9acad762d72689cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:30.317508 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-ae6ce111-d947-485f-a223-bd0ddf6d6865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:30.394564 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 956/1909] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:30.399815 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2a51b-0523-4a26-a6f8-10a788e7b8ad admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:30.481457 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2a51b-0523-4a26-a6f8-10a788e7b8ad admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:30.487265 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b81584-f6a0-4cd1-a2a3-ab6940b40ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:30.487872 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b81584-f6a0-4cd1-a2a3-ab6940b40ccc admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:30.488165 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b81584-f6a0-4cd1-a2a3-ab6940b40ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:30.488423 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b81584-f6a0-4cd1-a2a3-ab6940b40ccc admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:30.489127 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b81584-f6a0-4cd1-a2a3-ab6940b40ccc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:30.491344 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b81584-f6a0-4cd1-a2a3-ab6940b40ccc admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:30.520519 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 954/1910] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:11:30 2022] POST /identity/v3/projects => generated 381 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:30.524954 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ce111-d947-485f-a223-bd0ddf6d6865 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:30.535658 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ce111-d947-485f-a223-bd0ddf6d6865 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:30.560078 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:30.607436 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba811f0-9851-48af-abeb-07831b50162f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:30.608132 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba811f0-9851-48af-abeb-07831b50162f admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:30.608479 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba811f0-9851-48af-abeb-07831b50162f admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:30.608965 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba811f0-9851-48af-abeb-07831b50162f admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:30.609616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba811f0-9851-48af-abeb-07831b50162f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:30.611746 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba811f0-9851-48af-abeb-07831b50162f admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:30.645949 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 957/1911] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:11:30 2022] POST /identity/v3/users => generated 297 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:30.648030 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b81584-f6a0-4cd1-a2a3-ab6940b40ccc admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:30.658280 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b81584-f6a0-4cd1-a2a3-ab6940b40ccc admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:30.663026 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f831426-a2b3-4405-a3da-e4557fd78168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:30.663655 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f831426-a2b3-4405-a3da-e4557fd78168 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:30.663950 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f831426-a2b3-4405-a3da-e4557fd78168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:30.664223 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f831426-a2b3-4405-a3da-e4557fd78168 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:30.664899 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f831426-a2b3-4405-a3da-e4557fd78168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:30.666428 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f831426-a2b3-4405-a3da-e4557fd78168 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:30.677794 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 955/1912] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:30.682302 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba811f0-9851-48af-abeb-07831b50162f admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:30.692889 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba811f0-9851-48af-abeb-07831b50162f admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:30.698638 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aad430-ace3-4606-8bd1-04cbfc86c9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:30.699433 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aad430-ace3-4606-8bd1-04cbfc86c9d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:30.699712 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aad430-ace3-4606-8bd1-04cbfc86c9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:30.700124 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aad430-ace3-4606-8bd1-04cbfc86c9d6 admin admin] PATH_INFO: `/v3/projects/9c3eb5c3234f4b91b7887527487d1f63/users/b5ad7030679c4e7381fa30d70c2d6fe6/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:30.717198 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aad430-ace3-4606-8bd1-04cbfc86c9d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c3eb5c3234f4b91b7887527487d1f63, user_id=b5ad7030679c4e7381fa30d70c2d6fe6, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:30.720416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aad430-ace3-4606-8bd1-04cbfc86c9d6 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:30.734530 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 958/1913] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:30 2022] PUT /identity/v3/projects/9c3eb5c3234f4b91b7887527487d1f63/users/b5ad7030679c4e7381fa30d70c2d6fe6/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:30.738722 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f831426-a2b3-4405-a3da-e4557fd78168 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:30.748611 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f831426-a2b3-4405-a3da-e4557fd78168 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:30.792283 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aad430-ace3-4606-8bd1-04cbfc86c9d6 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:30.794086 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-79aad430-ace3-4606-8bd1-04cbfc86c9d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:30.813060 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25109df2-28f2-40ae-9271-d91ae88ed76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:30.813682 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25109df2-28f2-40ae-9271-d91ae88ed76c admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:30.813965 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25109df2-28f2-40ae-9271-d91ae88ed76c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:30.814691 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25109df2-28f2-40ae-9271-d91ae88ed76c admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:30.815474 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25109df2-28f2-40ae-9271-d91ae88ed76c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:30.817188 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25109df2-28f2-40ae-9271-d91ae88ed76c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:30.828885 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 956/1914] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:30 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:30.833131 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25109df2-28f2-40ae-9271-d91ae88ed76c admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:30.843243 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25109df2-28f2-40ae-9271-d91ae88ed76c admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:30.848457 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903df7c8-059f-44e3-8996-e4af844f127e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:30.849090 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903df7c8-059f-44e3-8996-e4af844f127e admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:30.849668 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903df7c8-059f-44e3-8996-e4af844f127e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:30.849952 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903df7c8-059f-44e3-8996-e4af844f127e admin admin] PATH_INFO: `/v3/projects/9c3eb5c3234f4b91b7887527487d1f63/users/b5ad7030679c4e7381fa30d70c2d6fe6/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:30.851933 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903df7c8-059f-44e3-8996-e4af844f127e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c3eb5c3234f4b91b7887527487d1f63, user_id=b5ad7030679c4e7381fa30d70c2d6fe6, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:30.854581 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903df7c8-059f-44e3-8996-e4af844f127e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:30.860650 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:30.864122 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aad430-ace3-4606-8bd1-04cbfc86c9d6 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:30.867622 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 957/1915] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:30 2022] PUT /identity/v3/projects/9c3eb5c3234f4b91b7887527487d1f63/users/b5ad7030679c4e7381fa30d70c2d6fe6/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:30.879576 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198ee3c-7e72-49a3-9808-999f24916b53 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:30.879859 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198ee3c-7e72-49a3-9808-999f24916b53 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:30.880128 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198ee3c-7e72-49a3-9808-999f24916b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:30.909700 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-a198ee3c-7e72-49a3-9808-999f24916b53 None None] MFA Rules not processed for user `b5ad7030679c4e7381fa30d70c2d6fe6`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:30.910875 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a198ee3c-7e72-49a3-9808-999f24916b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:30.933963 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20972bc-1cea-4c16-93e1-84dc3cc47054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:30.934606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20972bc-1cea-4c16-93e1-84dc3cc47054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:30.934881 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20972bc-1cea-4c16-93e1-84dc3cc47054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:30.935186 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20972bc-1cea-4c16-93e1-84dc3cc47054 admin admin] PATH_INFO: `/v3/users/b0abac9ba591401aacd7326f4b50ce38` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:30.936334 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20972bc-1cea-4c16-93e1-84dc3cc47054 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0abac9ba591401aacd7326f4b50ce38)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:30.938280 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20972bc-1cea-4c16-93e1-84dc3cc47054 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:30.977317 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:30.985052 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 958/1916] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:30 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:11:30.987027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-b20972bc-1cea-4c16-93e1-84dc3cc47054 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0abac9ba591401aacd7326f4b50ce38'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:30.994771 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-b20972bc-1cea-4c16-93e1-84dc3cc47054 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0abac9ba591401aacd7326f4b50ce38'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:30.995475 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-b20972bc-1cea-4c16-93e1-84dc3cc47054 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0abac9ba591401aacd7326f4b50ce38'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:30.996083 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-b20972bc-1cea-4c16-93e1-84dc3cc47054 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0abac9ba591401aacd7326f4b50ce38'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:30.997870 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198ee3c-7e72-49a3-9808-999f24916b53 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:30.999300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a198ee3c-7e72-49a3-9808-999f24916b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:31.003453 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-b20972bc-1cea-4c16-93e1-84dc3cc47054 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0abac9ba591401aacd7326f4b50ce38'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:31.021379 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 959/1917] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:11:30 2022] DELETE /identity/v3/users/b0abac9ba591401aacd7326f4b50ce38 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:11:31.025588 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74c539-4a89-4308-8f6d-6d4c2cad702d None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:31.025875 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74c539-4a89-4308-8f6d-6d4c2cad702d None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:31.026147 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74c539-4a89-4308-8f6d-6d4c2cad702d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:31.044307 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-7f74c539-4a89-4308-8f6d-6d4c2cad702d None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:31.045495 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-7f74c539-4a89-4308-8f6d-6d4c2cad702d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:31.057543 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198ee3c-7e72-49a3-9808-999f24916b53 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:31.113871 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 960/1918] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:11:31.121076 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733f7ac-c3f2-4eeb-9499-f526ddbf90c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:31.121681 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733f7ac-c3f2-4eeb-9499-f526ddbf90c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:31.121956 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733f7ac-c3f2-4eeb-9499-f526ddbf90c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:31.122222 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733f7ac-c3f2-4eeb-9499-f526ddbf90c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:31.124009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f733f7ac-c3f2-4eeb-9499-f526ddbf90c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:31.125576 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f74c539-4a89-4308-8f6d-6d4c2cad702d None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:31.126991 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-7f74c539-4a89-4308-8f6d-6d4c2cad702d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:31.190100 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:31.192848 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f74c539-4a89-4308-8f6d-6d4c2cad702d None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:31.197755 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e1f6c-ae69-4ead-a9ae-af970bf07700 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:31.198365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e1f6c-ae69-4ead-a9ae-af970bf07700 service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:31.198837 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e1f6c-ae69-4ead-a9ae-af970bf07700 service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:31.199269 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e1f6c-ae69-4ead-a9ae-af970bf07700 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:31.204211 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733f7ac-c3f2-4eeb-9499-f526ddbf90c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:31.205725 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733f7ac-c3f2-4eeb-9499-f526ddbf90c4 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:31.208926 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e1f6c-ae69-4ead-a9ae-af970bf07700 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:31.210692 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e1f6c-ae69-4ead-a9ae-af970bf07700 service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:31.216928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 959/1919] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:30 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 13 13:11:31.221661 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 961/1920] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:11:31.515997 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733f7ac-c3f2-4eeb-9499-f526ddbf90c4 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:31.518613 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f733f7ac-c3f2-4eeb-9499-f526ddbf90c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:31.541430 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733f7ac-c3f2-4eeb-9499-f526ddbf90c4 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:31.546743 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f929665-d4cb-499a-b0cb-45625920a449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:31.548558 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f929665-d4cb-499a-b0cb-45625920a449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:31.548855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f929665-d4cb-499a-b0cb-45625920a449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:31.549149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f929665-d4cb-499a-b0cb-45625920a449 admin admin] PATH_INFO: `/v3/projects/1a2a63ae7c094355a8b52dcec4a2239c` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:31.550310 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f929665-d4cb-499a-b0cb-45625920a449 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a2a63ae7c094355a8b52dcec4a2239c)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:31.553324 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f929665-d4cb-499a-b0cb-45625920a449 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:31.613540 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2f929665-d4cb-499a-b0cb-45625920a449 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a2a63ae7c094355a8b52dcec4a2239c', 'request_id': 'req-2f929665-d4cb-499a-b0cb-45625920a449'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:31.619768 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2f929665-d4cb-499a-b0cb-45625920a449 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a2a63ae7c094355a8b52dcec4a2239c', 'request_id': 'req-2f929665-d4cb-499a-b0cb-45625920a449'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:31.626191 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2f929665-d4cb-499a-b0cb-45625920a449 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a2a63ae7c094355a8b52dcec4a2239c', 'request_id': 'req-2f929665-d4cb-499a-b0cb-45625920a449'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:31.626363 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e1f6c-ae69-4ead-a9ae-af970bf07700 service neutron] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:31.627778 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-345e1f6c-ae69-4ead-a9ae-af970bf07700 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:31.639976 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2f929665-d4cb-499a-b0cb-45625920a449 admin admin] The token cache is being invalidate because project 1a2a63ae7c094355a8b52dcec4a2239c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:11:31.640302 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2f929665-d4cb-499a-b0cb-45625920a449 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:31.640890 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2f929665-d4cb-499a-b0cb-45625920a449 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:31.641003 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e1f6c-ae69-4ead-a9ae-af970bf07700 service neutron] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:31.642469 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 960/1921] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:11:31 2022] DELETE /identity/v3/projects/1a2a63ae7c094355a8b52dcec4a2239c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:31.646252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f929665-d4cb-499a-b0cb-45625920a449 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:31.648131 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2f929665-d4cb-499a-b0cb-45625920a449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:31.706448 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:31.715418 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528dc836-e8c9-4fe3-9bf7-955056f5c485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:31.716304 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528dc836-e8c9-4fe3-9bf7-955056f5c485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:31.716758 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528dc836-e8c9-4fe3-9bf7-955056f5c485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:31.717128 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528dc836-e8c9-4fe3-9bf7-955056f5c485 admin admin] PATH_INFO: `/v3/users/641dcf98932a45fe9acad762d72689cd` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:31.718473 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528dc836-e8c9-4fe3-9bf7-955056f5c485 admin admin] RBAC: Authorizing `identity:delete_user(user_id=641dcf98932a45fe9acad762d72689cd)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:31.719727 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f929665-d4cb-499a-b0cb-45625920a449 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:31.720436 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528dc836-e8c9-4fe3-9bf7-955056f5c485 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:31.724360 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31277207-f5bc-4a39-9b26-000e236f83b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:31.725592 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31277207-f5bc-4a39-9b26-000e236f83b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:31.725874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31277207-f5bc-4a39-9b26-000e236f83b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:31.726149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31277207-f5bc-4a39-9b26-000e236f83b0 admin admin] PATH_INFO: `/v3/users/9c19a42c75514b93a6363928a9a8914d` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:31.727303 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31277207-f5bc-4a39-9b26-000e236f83b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c19a42c75514b93a6363928a9a8914d)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:31.729322 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31277207-f5bc-4a39-9b26-000e236f83b0 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:31.775239 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-528dc836-e8c9-4fe3-9bf7-955056f5c485 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '641dcf98932a45fe9acad762d72689cd'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:31.777321 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-31277207-f5bc-4a39-9b26-000e236f83b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c19a42c75514b93a6363928a9a8914d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:31.782173 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-528dc836-e8c9-4fe3-9bf7-955056f5c485 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '641dcf98932a45fe9acad762d72689cd'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:31.782840 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-528dc836-e8c9-4fe3-9bf7-955056f5c485 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '641dcf98932a45fe9acad762d72689cd'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:31.783470 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-528dc836-e8c9-4fe3-9bf7-955056f5c485 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '641dcf98932a45fe9acad762d72689cd'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:31.784641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-31277207-f5bc-4a39-9b26-000e236f83b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c19a42c75514b93a6363928a9a8914d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:31.785453 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-31277207-f5bc-4a39-9b26-000e236f83b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c19a42c75514b93a6363928a9a8914d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:31.792349 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-528dc836-e8c9-4fe3-9bf7-955056f5c485 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '641dcf98932a45fe9acad762d72689cd'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:31.794677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-31277207-f5bc-4a39-9b26-000e236f83b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c19a42c75514b93a6363928a9a8914d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:31.803723 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:31.809555 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-31277207-f5bc-4a39-9b26-000e236f83b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c19a42c75514b93a6363928a9a8914d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:31.811764 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 962/1922] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:11:31 2022] DELETE /identity/v3/users/641dcf98932a45fe9acad762d72689cd => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:31.816071 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347f430-6fad-456e-acbb-4330d1ee66f5 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:31.816445 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347f430-6fad-456e-acbb-4330d1ee66f5 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:31.816872 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347f430-6fad-456e-acbb-4330d1ee66f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:31.819564 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 961/1923] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:11:31 2022] DELETE /identity/v3/users/9c19a42c75514b93a6363928a9a8914d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:11:31.836853 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-c347f430-6fad-456e-acbb-4330d1ee66f5 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:31.837966 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-c347f430-6fad-456e-acbb-4330d1ee66f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:31.896885 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 963/1924] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:11:31.906612 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31277207-f5bc-4a39-9b26-000e236f83b0 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:31.908093 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-31277207-f5bc-4a39-9b26-000e236f83b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:31.976122 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31277207-f5bc-4a39-9b26-000e236f83b0 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:31.980175 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7838f3-b9e3-420a-b06a-7f5736ddd63f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:31.980814 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7838f3-b9e3-420a-b06a-7f5736ddd63f service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:31.981536 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7838f3-b9e3-420a-b06a-7f5736ddd63f service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:31.981885 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7838f3-b9e3-420a-b06a-7f5736ddd63f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:31.992389 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7838f3-b9e3-420a-b06a-7f5736ddd63f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:31.993893 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7838f3-b9e3-420a-b06a-7f5736ddd63f service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:32.003689 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 962/1925] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:11:32.182820 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347f430-6fad-456e-acbb-4330d1ee66f5 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:32.195472 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-c347f430-6fad-456e-acbb-4330d1ee66f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:32.198231 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347f430-6fad-456e-acbb-4330d1ee66f5 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:32.214362 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8a789-3bdc-417d-9ce5-1919661e3eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:32.215435 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8a789-3bdc-417d-9ce5-1919661e3eda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:32.215952 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8a789-3bdc-417d-9ce5-1919661e3eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:32.216400 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8a789-3bdc-417d-9ce5-1919661e3eda admin admin] PATH_INFO: `/v3/projects/e7657c3cfbec43248732f0881c46e33c` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:32.218000 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8a789-3bdc-417d-9ce5-1919661e3eda admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7657c3cfbec43248732f0881c46e33c)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:32.220281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8a789-3bdc-417d-9ce5-1919661e3eda admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:32.275846 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-9ef8a789-3bdc-417d-9ce5-1919661e3eda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7657c3cfbec43248732f0881c46e33c', 'request_id': 'req-9ef8a789-3bdc-417d-9ce5-1919661e3eda'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:32.281566 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-9ef8a789-3bdc-417d-9ce5-1919661e3eda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7657c3cfbec43248732f0881c46e33c', 'request_id': 'req-9ef8a789-3bdc-417d-9ce5-1919661e3eda'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:32.288134 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-9ef8a789-3bdc-417d-9ce5-1919661e3eda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7657c3cfbec43248732f0881c46e33c', 'request_id': 'req-9ef8a789-3bdc-417d-9ce5-1919661e3eda'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:32.297483 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:32.301263 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-9ef8a789-3bdc-417d-9ce5-1919661e3eda admin admin] The token cache is being invalidate because project e7657c3cfbec43248732f0881c46e33c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:11:32.302075 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-9ef8a789-3bdc-417d-9ce5-1919661e3eda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:32.303349 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-9ef8a789-3bdc-417d-9ce5-1919661e3eda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:32.305082 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 964/1926] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:11:32 2022] DELETE /identity/v3/projects/e7657c3cfbec43248732f0881c46e33c => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:32.310220 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d6aabe-1577-44be-b6b8-7f85e7e90833 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:32.310642 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d6aabe-1577-44be-b6b8-7f85e7e90833 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:32.311033 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d6aabe-1577-44be-b6b8-7f85e7e90833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:32.330947 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-75d6aabe-1577-44be-b6b8-7f85e7e90833 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:32.332179 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-75d6aabe-1577-44be-b6b8-7f85e7e90833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:32.381182 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8a789-3bdc-417d-9ce5-1919661e3eda admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:32.382651 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-9ef8a789-3bdc-417d-9ce5-1919661e3eda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:32.429479 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 963/1927] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:32.438156 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa5b82-bb30-4fb4-bf1c-90a005cb9702 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:32.438444 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa5b82-bb30-4fb4-bf1c-90a005cb9702 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:32.439446 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa5b82-bb30-4fb4-bf1c-90a005cb9702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:32.458723 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8caa5b82-bb30-4fb4-bf1c-90a005cb9702 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:32.460798 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8caa5b82-bb30-4fb4-bf1c-90a005cb9702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:32.475366 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 964/1928] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:11:32.479789 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa5b82-bb30-4fb4-bf1c-90a005cb9702 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:32.492848 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa5b82-bb30-4fb4-bf1c-90a005cb9702 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:32.493607 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8a789-3bdc-417d-9ce5-1919661e3eda admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:32.499632 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab328fe-b8e0-4cec-99ce-6c80cf51f7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:32.502087 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4728e8c9-02b0-4eeb-9d93-62b59f57f9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:32.502709 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728e8c9-02b0-4eeb-9d93-62b59f57f9a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:32.502982 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728e8c9-02b0-4eeb-9d93-62b59f57f9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:32.503991 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728e8c9-02b0-4eeb-9d93-62b59f57f9a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:32.505017 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4728e8c9-02b0-4eeb-9d93-62b59f57f9a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:11:32.505017 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4728e8c9-02b0-4eeb-9d93-62b59f57f9a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:32.505287 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab328fe-b8e0-4cec-99ce-6c80cf51f7b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:32.505287 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab328fe-b8e0-4cec-99ce-6c80cf51f7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:32.505671 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab328fe-b8e0-4cec-99ce-6c80cf51f7b4 admin admin] PATH_INFO: `/v3/projects/03b0bc604e8e474ba837ed92efb7506c` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:32.506767 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4728e8c9-02b0-4eeb-9d93-62b59f57f9a4 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:32.507807 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab328fe-b8e0-4cec-99ce-6c80cf51f7b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03b0bc604e8e474ba837ed92efb7506c)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:32.510448 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab328fe-b8e0-4cec-99ce-6c80cf51f7b4 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:32.520875 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 965/1929] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:11:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 13:11:32.525024 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4728e8c9-02b0-4eeb-9d93-62b59f57f9a4 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:32.538125 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4728e8c9-02b0-4eeb-9d93-62b59f57f9a4 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:32.549407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a81c3-052f-49bb-8ffe-cdba78e7b520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:32.550684 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a81c3-052f-49bb-8ffe-cdba78e7b520 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:32.551144 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a81c3-052f-49bb-8ffe-cdba78e7b520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:32.551426 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a81c3-052f-49bb-8ffe-cdba78e7b520 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:32.552343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335a81c3-052f-49bb-8ffe-cdba78e7b520 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:32.554301 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335a81c3-052f-49bb-8ffe-cdba78e7b520 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:32.571077 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-3ab328fe-b8e0-4cec-99ce-6c80cf51f7b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03b0bc604e8e474ba837ed92efb7506c', 'request_id': 'req-3ab328fe-b8e0-4cec-99ce-6c80cf51f7b4'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:32.576845 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-3ab328fe-b8e0-4cec-99ce-6c80cf51f7b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03b0bc604e8e474ba837ed92efb7506c', 'request_id': 'req-3ab328fe-b8e0-4cec-99ce-6c80cf51f7b4'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:32.583087 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 966/1930] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:11:32 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:11:32.583242 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-3ab328fe-b8e0-4cec-99ce-6c80cf51f7b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03b0bc604e8e474ba837ed92efb7506c', 'request_id': 'req-3ab328fe-b8e0-4cec-99ce-6c80cf51f7b4'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:32.587549 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a81c3-052f-49bb-8ffe-cdba78e7b520 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:32.600861 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-3ab328fe-b8e0-4cec-99ce-6c80cf51f7b4 admin admin] The token cache is being invalidate because project 03b0bc604e8e474ba837ed92efb7506c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:11:32.601163 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-3ab328fe-b8e0-4cec-99ce-6c80cf51f7b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:32.601813 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-3ab328fe-b8e0-4cec-99ce-6c80cf51f7b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:32.603509 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 965/1931] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:11:32 2022] DELETE /identity/v3/projects/03b0bc604e8e474ba837ed92efb7506c => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:32.607333 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab328fe-b8e0-4cec-99ce-6c80cf51f7b4 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:32.608856 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-3ab328fe-b8e0-4cec-99ce-6c80cf51f7b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:32.643288 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:32.656549 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a81c3-052f-49bb-8ffe-cdba78e7b520 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:32.662717 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3aaa25-ab03-41ec-a1e6-266bc5104970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:32.663352 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3aaa25-ab03-41ec-a1e6-266bc5104970 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:32.663633 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3aaa25-ab03-41ec-a1e6-266bc5104970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:32.663909 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3aaa25-ab03-41ec-a1e6-266bc5104970 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:32.664543 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3aaa25-ab03-41ec-a1e6-266bc5104970 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:32.666495 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3aaa25-ab03-41ec-a1e6-266bc5104970 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:32.677498 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab328fe-b8e0-4cec-99ce-6c80cf51f7b4 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:32.682752 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c291c6f0-958c-4ef6-94ff-60f0bc48f2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:32.683838 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291c6f0-958c-4ef6-94ff-60f0bc48f2c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:32.684125 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291c6f0-958c-4ef6-94ff-60f0bc48f2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:32.684513 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291c6f0-958c-4ef6-94ff-60f0bc48f2c0 admin admin] PATH_INFO: `/v3/users/b5ad7030679c4e7381fa30d70c2d6fe6` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:32.685537 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c291c6f0-958c-4ef6-94ff-60f0bc48f2c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5ad7030679c4e7381fa30d70c2d6fe6)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:32.687673 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 967/1932] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:11:32 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:11:32.690667 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c291c6f0-958c-4ef6-94ff-60f0bc48f2c0 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:32.691707 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3aaa25-ab03-41ec-a1e6-266bc5104970 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:32.694254 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-bc3aaa25-ab03-41ec-a1e6-266bc5104970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:32.706123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3aaa25-ab03-41ec-a1e6-266bc5104970 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:32.711368 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c585bb7-89d0-48ca-bfec-ab9694e98240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:32.711962 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c585bb7-89d0-48ca-bfec-ab9694e98240 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:32.712237 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c585bb7-89d0-48ca-bfec-ab9694e98240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:32.712605 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c585bb7-89d0-48ca-bfec-ab9694e98240 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:32.713208 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c585bb7-89d0-48ca-bfec-ab9694e98240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:32.714686 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c585bb7-89d0-48ca-bfec-ab9694e98240 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:32.727094 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 968/1933] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:11:32.731911 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c585bb7-89d0-48ca-bfec-ab9694e98240 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:32.733171 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:32.744573 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c585bb7-89d0-48ca-bfec-ab9694e98240 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:32.749149 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c291c6f0-958c-4ef6-94ff-60f0bc48f2c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5ad7030679c4e7381fa30d70c2d6fe6'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:32.763425 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c7d0ea-9ec0-4b19-a1cd-aa17a412e30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:32.763425 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c7d0ea-9ec0-4b19-a1cd-aa17a412e30d admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:32.763425 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c7d0ea-9ec0-4b19-a1cd-aa17a412e30d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:32.763425 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c7d0ea-9ec0-4b19-a1cd-aa17a412e30d admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:32.763830 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c7d0ea-9ec0-4b19-a1cd-aa17a412e30d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:32.763830 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c7d0ea-9ec0-4b19-a1cd-aa17a412e30d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:32.763983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c291c6f0-958c-4ef6-94ff-60f0bc48f2c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5ad7030679c4e7381fa30d70c2d6fe6'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:32.763983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c291c6f0-958c-4ef6-94ff-60f0bc48f2c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5ad7030679c4e7381fa30d70c2d6fe6'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:32.763983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c291c6f0-958c-4ef6-94ff-60f0bc48f2c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5ad7030679c4e7381fa30d70c2d6fe6'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:32.772589 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c291c6f0-958c-4ef6-94ff-60f0bc48f2c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5ad7030679c4e7381fa30d70c2d6fe6'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:32.775700 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 969/1934] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:32 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:11:32.779789 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c7d0ea-9ec0-4b19-a1cd-aa17a412e30d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:32.781488 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-73c7d0ea-9ec0-4b19-a1cd-aa17a412e30d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:32.790326 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 966/1935] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:11:32 2022] DELETE /identity/v3/users/b5ad7030679c4e7381fa30d70c2d6fe6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:11:32.794271 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c7d0ea-9ec0-4b19-a1cd-aa17a412e30d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:32.855378 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7649e8d3-eb6f-4603-b350-675159295596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:32.856089 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7649e8d3-eb6f-4603-b350-675159295596 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:32.856428 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7649e8d3-eb6f-4603-b350-675159295596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:32.856646 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7649e8d3-eb6f-4603-b350-675159295596 admin admin] PATH_INFO: `/v3/projects/5bf9fa02077841f4a6ec28f62f61de36/users/4e99c80ce2ab4ad98c0d9dec0a151cd6/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:32.873885 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7649e8d3-eb6f-4603-b350-675159295596 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bf9fa02077841f4a6ec28f62f61de36, user_id=4e99c80ce2ab4ad98c0d9dec0a151cd6, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:32.876626 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7649e8d3-eb6f-4603-b350-675159295596 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:32.890365 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 970/1936] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:32 2022] PUT /identity/v3/projects/5bf9fa02077841f4a6ec28f62f61de36/users/4e99c80ce2ab4ad98c0d9dec0a151cd6/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:11:32.901481 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2d48e-631d-45e0-8334-ff3a8225eb36 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:32.901975 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2d48e-631d-45e0-8334-ff3a8225eb36 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:32.902065 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2d48e-631d-45e0-8334-ff3a8225eb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:32.942062 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-32a2d48e-631d-45e0-8334-ff3a8225eb36 None None] MFA Rules not processed for user `4e99c80ce2ab4ad98c0d9dec0a151cd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:32.943884 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-32a2d48e-631d-45e0-8334-ff3a8225eb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:33.011308 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 967/1937] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:33.015787 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7649e8d3-eb6f-4603-b350-675159295596 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:33.067257 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a2d48e-631d-45e0-8334-ff3a8225eb36 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:33.069531 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-32a2d48e-631d-45e0-8334-ff3a8225eb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:33.086887 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7649e8d3-eb6f-4603-b350-675159295596 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:33.091879 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0f56a-ba32-490c-826d-91b806c5c36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:33.092599 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0f56a-ba32-490c-826d-91b806c5c36f admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:33.092850 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0f56a-ba32-490c-826d-91b806c5c36f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:33.093126 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0f56a-ba32-490c-826d-91b806c5c36f admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:33.093776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef0f56a-ba32-490c-826d-91b806c5c36f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:33.095711 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef0f56a-ba32-490c-826d-91b806c5c36f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:33.122333 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 971/1938] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:11:33 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:11:33.126408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0f56a-ba32-490c-826d-91b806c5c36f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:33.140436 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a2d48e-631d-45e0-8334-ff3a8225eb36 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:33.159490 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:33.185871 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:33.194954 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0f56a-ba32-490c-826d-91b806c5c36f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:33.200270 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed65d6-06c5-4192-b4f1-33a391265f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:33.200897 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed65d6-06c5-4192-b4f1-33a391265f18 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:33.201173 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed65d6-06c5-4192-b4f1-33a391265f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:33.201438 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed65d6-06c5-4192-b4f1-33a391265f18 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:33.201979 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc75434a-2b6a-4ec5-87b3-764c07806008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:33.202085 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed65d6-06c5-4192-b4f1-33a391265f18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:33.202575 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75434a-2b6a-4ec5-87b3-764c07806008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:33.202945 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75434a-2b6a-4ec5-87b3-764c07806008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:33.203345 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75434a-2b6a-4ec5-87b3-764c07806008 admin admin] PATH_INFO: `/v3/projects/9c3eb5c3234f4b91b7887527487d1f63` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:33.204074 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed65d6-06c5-4192-b4f1-33a391265f18 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:33.204690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc75434a-2b6a-4ec5-87b3-764c07806008 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c3eb5c3234f4b91b7887527487d1f63)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:33.206969 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc75434a-2b6a-4ec5-87b3-764c07806008 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:33.226851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 972/1939] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:11:33 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:11:33.230682 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed65d6-06c5-4192-b4f1-33a391265f18 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:33.241210 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed65d6-06c5-4192-b4f1-33a391265f18 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:33.246453 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d719ed-76e8-4c15-a101-d09ec313a6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:33.247069 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d719ed-76e8-4c15-a101-d09ec313a6a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:33.247357 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d719ed-76e8-4c15-a101-d09ec313a6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:33.247629 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d719ed-76e8-4c15-a101-d09ec313a6a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:33.248272 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d719ed-76e8-4c15-a101-d09ec313a6a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:33.249840 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d719ed-76e8-4c15-a101-d09ec313a6a7 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:33.260338 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 973/1940] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:33.264624 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d719ed-76e8-4c15-a101-d09ec313a6a7 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:33.267214 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-dc75434a-2b6a-4ec5-87b3-764c07806008 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c3eb5c3234f4b91b7887527487d1f63', 'request_id': 'req-dc75434a-2b6a-4ec5-87b3-764c07806008'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:33.272740 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-dc75434a-2b6a-4ec5-87b3-764c07806008 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c3eb5c3234f4b91b7887527487d1f63', 'request_id': 'req-dc75434a-2b6a-4ec5-87b3-764c07806008'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:33.274226 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d719ed-76e8-4c15-a101-d09ec313a6a7 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:33.278797 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-dc75434a-2b6a-4ec5-87b3-764c07806008 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c3eb5c3234f4b91b7887527487d1f63', 'request_id': 'req-dc75434a-2b6a-4ec5-87b3-764c07806008'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:33.291477 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-dc75434a-2b6a-4ec5-87b3-764c07806008 admin admin] The token cache is being invalidate because project 9c3eb5c3234f4b91b7887527487d1f63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:11:33.291792 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-dc75434a-2b6a-4ec5-87b3-764c07806008 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:33.292462 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-dc75434a-2b6a-4ec5-87b3-764c07806008 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:33.294013 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 968/1941] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:11:33 2022] DELETE /identity/v3/projects/9c3eb5c3234f4b91b7887527487d1f63 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:11:33.298675 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2a477-8cba-410c-acb3-b1bed39715d7 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:33.298962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2a477-8cba-410c-acb3-b1bed39715d7 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:33.299261 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2a477-8cba-410c-acb3-b1bed39715d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:33.319738 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-97e2a477-8cba-410c-acb3-b1bed39715d7 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:33.320971 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-97e2a477-8cba-410c-acb3-b1bed39715d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:33.342763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6495db4-f079-42c6-9429-47e34a3e1eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:33.343551 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6495db4-f079-42c6-9429-47e34a3e1eab admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:33.345076 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6495db4-f079-42c6-9429-47e34a3e1eab admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:33.345353 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6495db4-f079-42c6-9429-47e34a3e1eab admin admin] PATH_INFO: `/v3/projects/8c51cbf8c85c46b59db522cd25eebbd9/users/a4b15507ed8148aaafe644e650836cbb/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:33.359696 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6495db4-f079-42c6-9429-47e34a3e1eab admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c51cbf8c85c46b59db522cd25eebbd9, user_id=a4b15507ed8148aaafe644e650836cbb, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:33.362703 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6495db4-f079-42c6-9429-47e34a3e1eab admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:33.375379 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 974/1942] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:33 2022] PUT /identity/v3/projects/8c51cbf8c85c46b59db522cd25eebbd9/users/a4b15507ed8148aaafe644e650836cbb/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:11:33.379307 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6495db4-f079-42c6-9429-47e34a3e1eab admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:33.381577 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b6495db4-f079-42c6-9429-47e34a3e1eab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:33.384563 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 969/1943] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:11:33.391919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf972e-ceb0-4040-8feb-a83e8663df8c None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:33.392205 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf972e-ceb0-4040-8feb-a83e8663df8c None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:33.392479 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf972e-ceb0-4040-8feb-a83e8663df8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:33.411687 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-6adf972e-ceb0-4040-8feb-a83e8663df8c None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:33.412910 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-6adf972e-ceb0-4040-8feb-a83e8663df8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:33.444317 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6495db4-f079-42c6-9429-47e34a3e1eab admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:33.448937 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9095d-121f-4c1b-ac4e-c3f1402c01a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:33.449565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9095d-121f-4c1b-ac4e-c3f1402c01a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:33.449843 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9095d-121f-4c1b-ac4e-c3f1402c01a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:33.450114 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9095d-121f-4c1b-ac4e-c3f1402c01a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:33.450750 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb9095d-121f-4c1b-ac4e-c3f1402c01a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:33.452376 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb9095d-121f-4c1b-ac4e-c3f1402c01a0 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:33.461983 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 975/1944] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:11:33.466156 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9095d-121f-4c1b-ac4e-c3f1402c01a0 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:33.475374 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9095d-121f-4c1b-ac4e-c3f1402c01a0 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:33.475720 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 970/1945] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:33.479818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf972e-ceb0-4040-8feb-a83e8663df8c None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:33.480314 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ad403-96be-43cd-a0c0-d55b88ef19b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:33.480956 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ad403-96be-43cd-a0c0-d55b88ef19b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:33.481241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ad403-96be-43cd-a0c0-d55b88ef19b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:33.481510 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ad403-96be-43cd-a0c0-d55b88ef19b7 admin admin] PATH_INFO: `/v3/projects/8c51cbf8c85c46b59db522cd25eebbd9/users/a4b15507ed8148aaafe644e650836cbb/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:33.483430 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433ad403-96be-43cd-a0c0-d55b88ef19b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c51cbf8c85c46b59db522cd25eebbd9, user_id=a4b15507ed8148aaafe644e650836cbb, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:33.486048 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433ad403-96be-43cd-a0c0-d55b88ef19b7 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:33.488945 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf972e-ceb0-4040-8feb-a83e8663df8c None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:33.494089 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd4637-dd0d-4ca3-8b8e-598d3749ec83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:33.494704 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd4637-dd0d-4ca3-8b8e-598d3749ec83 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:33.494981 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd4637-dd0d-4ca3-8b8e-598d3749ec83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:33.495279 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd4637-dd0d-4ca3-8b8e-598d3749ec83 admin admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:33.495918 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cd4637-dd0d-4ca3-8b8e-598d3749ec83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:11:33.496221 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cd4637-dd0d-4ca3-8b8e-598d3749ec83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:33.496898 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 976/1946] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:33 2022] PUT /identity/v3/projects/8c51cbf8c85c46b59db522cd25eebbd9/users/a4b15507ed8148aaafe644e650836cbb/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:11:33.497937 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cd4637-dd0d-4ca3-8b8e-598d3749ec83 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:33.506961 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3f473-f083-4338-bb08-2c97adee0ffe None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:33.507344 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3f473-f083-4338-bb08-2c97adee0ffe None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:33.507621 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3f473-f083-4338-bb08-2c97adee0ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:33.513025 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 971/1947] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:11:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 13:11:33.513969 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd4637-dd0d-4ca3-8b8e-598d3749ec83 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:33.520005 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:33.524213 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd4637-dd0d-4ca3-8b8e-598d3749ec83 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:33.532176 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:33.535130 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-78c3f473-f083-4338-bb08-2c97adee0ffe None None] MFA Rules not processed for user `a4b15507ed8148aaafe644e650836cbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:33.537387 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-78c3f473-f083-4338-bb08-2c97adee0ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:33.594385 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0003538-56a5-40f6-9101-2d61293ca924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:33.595000 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0003538-56a5-40f6-9101-2d61293ca924 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:33.599281 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0003538-56a5-40f6-9101-2d61293ca924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:33.600198 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0003538-56a5-40f6-9101-2d61293ca924 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:33.601002 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0003538-56a5-40f6-9101-2d61293ca924 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:33.603081 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0003538-56a5-40f6-9101-2d61293ca924 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:33.611157 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 977/1948] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:33 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:11:33.622898 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3f473-f083-4338-bb08-2c97adee0ffe None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:33.624399 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-78c3f473-f083-4338-bb08-2c97adee0ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:33.640155 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 972/1949] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:11:33 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:11:33.644381 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0003538-56a5-40f6-9101-2d61293ca924 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:33.693892 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3f473-f083-4338-bb08-2c97adee0ffe None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:33.710499 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0003538-56a5-40f6-9101-2d61293ca924 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:33.715550 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a1002-d6f0-48a9-a572-74992d97fd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:33.716276 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a1002-d6f0-48a9-a572-74992d97fd93 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:33.716470 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a1002-d6f0-48a9-a572-74992d97fd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:33.727646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a1002-d6f0-48a9-a572-74992d97fd93 admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:33.727646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a1002-d6f0-48a9-a572-74992d97fd93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:33.727646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a1002-d6f0-48a9-a572-74992d97fd93 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:33.745061 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 973/1950] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:11:33 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:33.749106 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a1002-d6f0-48a9-a572-74992d97fd93 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:33.756748 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db9777-6d48-4ff6-87a2-4a402b79c0b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:33.757367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db9777-6d48-4ff6-87a2-4a402b79c0b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:33.757644 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db9777-6d48-4ff6-87a2-4a402b79c0b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:33.757914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db9777-6d48-4ff6-87a2-4a402b79c0b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:33.761664 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a1002-d6f0-48a9-a572-74992d97fd93 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:33.780791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d60124-c043-4c61-b9d9-ab078759c2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:33.783422 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d60124-c043-4c61-b9d9-ab078759c2a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:33.785755 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d60124-c043-4c61-b9d9-ab078759c2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:33.786126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d60124-c043-4c61-b9d9-ab078759c2a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:33.787399 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d60124-c043-4c61-b9d9-ab078759c2a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:33.791873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d60124-c043-4c61-b9d9-ab078759c2a6 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:33.808640 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 974/1951] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:11:33.810805 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d60124-c043-4c61-b9d9-ab078759c2a6 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:33.829546 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d60124-c043-4c61-b9d9-ab078759c2a6 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:33.834806 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78db9777-6d48-4ff6-87a2-4a402b79c0b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:33.836322 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78db9777-6d48-4ff6-87a2-4a402b79c0b6 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:33.841192 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba03799-b935-4bbf-944a-2b75205a14e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:33.842273 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba03799-b935-4bbf-944a-2b75205a14e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:33.842749 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba03799-b935-4bbf-944a-2b75205a14e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:33.843347 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba03799-b935-4bbf-944a-2b75205a14e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:33.844115 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba03799-b935-4bbf-944a-2b75205a14e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:33.845994 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba03799-b935-4bbf-944a-2b75205a14e5 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:33.850071 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 978/1952] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:33 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 13 13:11:33.859137 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 975/1953] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:33.863867 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db9777-6d48-4ff6-87a2-4a402b79c0b6 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:33.870897 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:33.876191 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db9777-6d48-4ff6-87a2-4a402b79c0b6 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:33.882124 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c4831-6cf2-49ca-80e2-1f82c63a2997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:33.882853 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c4831-6cf2-49ca-80e2-1f82c63a2997 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:33.883253 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c4831-6cf2-49ca-80e2-1f82c63a2997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:33.883604 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c4831-6cf2-49ca-80e2-1f82c63a2997 admin admin] PATH_INFO: `/v3/projects/ec8572a8f5484f5ca69cc125fc6025e9/users/208c0e97f7fe44ce8712580db0677bab/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:33.902208 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c4831-6cf2-49ca-80e2-1f82c63a2997 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec8572a8f5484f5ca69cc125fc6025e9, user_id=208c0e97f7fe44ce8712580db0677bab, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:33.904337 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c4831-6cf2-49ca-80e2-1f82c63a2997 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:33.919533 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 979/1954] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:33 2022] PUT /identity/v3/projects/ec8572a8f5484f5ca69cc125fc6025e9/users/208c0e97f7fe44ce8712580db0677bab/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:33.926997 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117c0db-bb58-4650-8502-5ffa4d9dd5a9 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:33.927551 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117c0db-bb58-4650-8502-5ffa4d9dd5a9 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:33.927811 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117c0db-bb58-4650-8502-5ffa4d9dd5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:33.935324 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c4831-6cf2-49ca-80e2-1f82c63a2997 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:33.954334 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-7117c0db-bb58-4650-8502-5ffa4d9dd5a9 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:33.955659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-7117c0db-bb58-4650-8502-5ffa4d9dd5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:34.005173 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c4831-6cf2-49ca-80e2-1f82c63a2997 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:34.010103 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420b39c-be55-4d1e-94c7-163e4c5091d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:34.010738 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420b39c-be55-4d1e-94c7-163e4c5091d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:34.011017 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420b39c-be55-4d1e-94c7-163e4c5091d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:34.011317 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420b39c-be55-4d1e-94c7-163e4c5091d5 admin admin] PATH_INFO: `/v3/users/4e99c80ce2ab4ad98c0d9dec0a151cd6` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:34.012455 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2420b39c-be55-4d1e-94c7-163e4c5091d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e99c80ce2ab4ad98c0d9dec0a151cd6)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:34.014395 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2420b39c-be55-4d1e-94c7-163e4c5091d5 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:34.016196 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:34.026798 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 976/1955] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:34.036246 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117c0db-bb58-4650-8502-5ffa4d9dd5a9 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:34.038420 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-7117c0db-bb58-4650-8502-5ffa4d9dd5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:34.073805 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2420b39c-be55-4d1e-94c7-163e4c5091d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e99c80ce2ab4ad98c0d9dec0a151cd6'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:34.081026 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2420b39c-be55-4d1e-94c7-163e4c5091d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e99c80ce2ab4ad98c0d9dec0a151cd6'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:34.081730 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2420b39c-be55-4d1e-94c7-163e4c5091d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e99c80ce2ab4ad98c0d9dec0a151cd6'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:34.091930 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2420b39c-be55-4d1e-94c7-163e4c5091d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e99c80ce2ab4ad98c0d9dec0a151cd6'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:34.102813 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117c0db-bb58-4650-8502-5ffa4d9dd5a9 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:34.106912 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb68a3-d0cc-426b-840e-1918f85526a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:34.107561 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb68a3-d0cc-426b-840e-1918f85526a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:34.107849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb68a3-d0cc-426b-840e-1918f85526a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:34.108116 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb68a3-d0cc-426b-840e-1918f85526a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:34.109276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2420b39c-be55-4d1e-94c7-163e4c5091d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e99c80ce2ab4ad98c0d9dec0a151cd6'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:34.110119 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-defb68a3-d0cc-426b-840e-1918f85526a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:34.115671 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 980/1956] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:11:33 2022] DELETE /identity/v3/users/4e99c80ce2ab4ad98c0d9dec0a151cd6 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:11:34.120275 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8013799-9eca-4e99-978b-5a3d1cae9b3f None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:34.120735 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8013799-9eca-4e99-978b-5a3d1cae9b3f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:34.121119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8013799-9eca-4e99-978b-5a3d1cae9b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:34.126801 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb68a3-d0cc-426b-840e-1918f85526a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:34.128382 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb68a3-d0cc-426b-840e-1918f85526a0 service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:34.139118 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 977/1957] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:11:34.145955 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d8013799-9eca-4e99-978b-5a3d1cae9b3f None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:34.147856 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d8013799-9eca-4e99-978b-5a3d1cae9b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:34.202986 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:34.218525 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 981/1958] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:11:34.232507 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb68a3-d0cc-426b-840e-1918f85526a0 service neutron] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:34.234392 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-defb68a3-d0cc-426b-840e-1918f85526a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:34.249528 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb68a3-d0cc-426b-840e-1918f85526a0 service neutron] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:34.342446 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c99965-e0cc-4534-86ba-c709d81b208e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:34.343158 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c99965-e0cc-4534-86ba-c709d81b208e service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:34.343563 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c99965-e0cc-4534-86ba-c709d81b208e service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:34.343982 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c99965-e0cc-4534-86ba-c709d81b208e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:34.369914 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c99965-e0cc-4534-86ba-c709d81b208e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:34.371464 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c99965-e0cc-4534-86ba-c709d81b208e service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:34.381896 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 978/1959] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:11:34.690038 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8013799-9eca-4e99-978b-5a3d1cae9b3f None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:34.691511 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d8013799-9eca-4e99-978b-5a3d1cae9b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:34.704184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8013799-9eca-4e99-978b-5a3d1cae9b3f None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:34.708765 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7615e14a-ddcf-453e-87a9-d4e85f1e4877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:34.709408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615e14a-ddcf-453e-87a9-d4e85f1e4877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:34.709672 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615e14a-ddcf-453e-87a9-d4e85f1e4877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:34.709933 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7615e14a-ddcf-453e-87a9-d4e85f1e4877 admin admin] PATH_INFO: `/v3/projects/5bf9fa02077841f4a6ec28f62f61de36` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:34.711073 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7615e14a-ddcf-453e-87a9-d4e85f1e4877 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bf9fa02077841f4a6ec28f62f61de36)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:34.712982 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7615e14a-ddcf-453e-87a9-d4e85f1e4877 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:34.764638 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7615e14a-ddcf-453e-87a9-d4e85f1e4877 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bf9fa02077841f4a6ec28f62f61de36', 'request_id': 'req-7615e14a-ddcf-453e-87a9-d4e85f1e4877'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:34.771212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7615e14a-ddcf-453e-87a9-d4e85f1e4877 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bf9fa02077841f4a6ec28f62f61de36', 'request_id': 'req-7615e14a-ddcf-453e-87a9-d4e85f1e4877'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:34.778776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7615e14a-ddcf-453e-87a9-d4e85f1e4877 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bf9fa02077841f4a6ec28f62f61de36', 'request_id': 'req-7615e14a-ddcf-453e-87a9-d4e85f1e4877'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:34.792388 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7615e14a-ddcf-453e-87a9-d4e85f1e4877 admin admin] The token cache is being invalidate because project 5bf9fa02077841f4a6ec28f62f61de36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:11:34.792827 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7615e14a-ddcf-453e-87a9-d4e85f1e4877 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:34.794043 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7615e14a-ddcf-453e-87a9-d4e85f1e4877 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:34.798965 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 982/1960] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:11:34 2022] DELETE /identity/v3/projects/5bf9fa02077841f4a6ec28f62f61de36 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:34.800186 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c99965-e0cc-4534-86ba-c709d81b208e service neutron] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:34.811304 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c99965-e0cc-4534-86ba-c709d81b208e service neutron] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:34.879403 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b4678-8fbe-4fec-88f1-e0b47b333502 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:34.879531 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b4678-8fbe-4fec-88f1-e0b47b333502 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:34.879724 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b4678-8fbe-4fec-88f1-e0b47b333502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:34.890001 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179fcfd-b496-4643-a701-5dd0c0640dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:34.890962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179fcfd-b496-4643-a701-5dd0c0640dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:34.891333 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179fcfd-b496-4643-a701-5dd0c0640dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:34.891704 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179fcfd-b496-4643-a701-5dd0c0640dcb admin admin] PATH_INFO: `/v3/users/a4b15507ed8148aaafe644e650836cbb` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:34.893428 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9179fcfd-b496-4643-a701-5dd0c0640dcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4b15507ed8148aaafe644e650836cbb)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:34.897374 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9179fcfd-b496-4643-a701-5dd0c0640dcb admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:34.940769 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-604b4678-8fbe-4fec-88f1-e0b47b333502 None None] MFA Rules not processed for user `208c0e97f7fe44ce8712580db0677bab`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:34.942041 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-604b4678-8fbe-4fec-88f1-e0b47b333502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:34.963475 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:34.966094 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-9179fcfd-b496-4643-a701-5dd0c0640dcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4b15507ed8148aaafe644e650836cbb'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:34.978421 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-9179fcfd-b496-4643-a701-5dd0c0640dcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4b15507ed8148aaafe644e650836cbb'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:34.979792 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-9179fcfd-b496-4643-a701-5dd0c0640dcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4b15507ed8148aaafe644e650836cbb'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:34.980814 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-9179fcfd-b496-4643-a701-5dd0c0640dcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4b15507ed8148aaafe644e650836cbb'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:34.991931 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-9179fcfd-b496-4643-a701-5dd0c0640dcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4b15507ed8148aaafe644e650836cbb'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:35.014105 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 979/1961] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:11:34 2022] DELETE /identity/v3/users/a4b15507ed8148aaafe644e650836cbb => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:35.017125 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 983/1962] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:35.021352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179fcfd-b496-4643-a701-5dd0c0640dcb admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:35.022741 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-9179fcfd-b496-4643-a701-5dd0c0640dcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:35.102167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179fcfd-b496-4643-a701-5dd0c0640dcb admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:35.107544 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b06e4f-32d3-4439-bf1e-ac0261a321ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:35.108230 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b06e4f-32d3-4439-bf1e-ac0261a321ac admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:35.108577 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b06e4f-32d3-4439-bf1e-ac0261a321ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:35.108914 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b06e4f-32d3-4439-bf1e-ac0261a321ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:35.109615 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b06e4f-32d3-4439-bf1e-ac0261a321ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:35.111704 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b06e4f-32d3-4439-bf1e-ac0261a321ac admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:35.143094 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 980/1963] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:11:35 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:35.144695 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b4678-8fbe-4fec-88f1-e0b47b333502 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:35.155493 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b4678-8fbe-4fec-88f1-e0b47b333502 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:35.214870 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d2726-7dee-4541-bee2-3e3fe5f923b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:35.215514 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d2726-7dee-4541-bee2-3e3fe5f923b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:35.215793 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d2726-7dee-4541-bee2-3e3fe5f923b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:35.216067 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d2726-7dee-4541-bee2-3e3fe5f923b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:35.216700 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d2726-7dee-4541-bee2-3e3fe5f923b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:35.218618 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d2726-7dee-4541-bee2-3e3fe5f923b6 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:35.243849 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 984/1964] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:11:35 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:35.245285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b06e4f-32d3-4439-bf1e-ac0261a321ac admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:35.255962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b06e4f-32d3-4439-bf1e-ac0261a321ac admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:35.260688 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bad896b-f185-44f6-9481-aa77c9fb9d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:35.261289 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad896b-f185-44f6-9481-aa77c9fb9d17 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:35.261657 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad896b-f185-44f6-9481-aa77c9fb9d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:35.261929 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad896b-f185-44f6-9481-aa77c9fb9d17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:35.262564 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bad896b-f185-44f6-9481-aa77c9fb9d17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:35.264212 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bad896b-f185-44f6-9481-aa77c9fb9d17 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:35.275098 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 981/1965] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:35.279416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d2726-7dee-4541-bee2-3e3fe5f923b6 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:35.284910 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:35.290986 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d2726-7dee-4541-bee2-3e3fe5f923b6 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:35.296914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42794580-2920-414d-99d1-d1e7caa455a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:35.297696 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42794580-2920-414d-99d1-d1e7caa455a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:35.298241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42794580-2920-414d-99d1-d1e7caa455a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:35.298534 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42794580-2920-414d-99d1-d1e7caa455a9 admin admin] PATH_INFO: `/v3/projects/aada7626dfa44f5ebb843842c5859b51/users/f8b6d604fc594479a076d38a669ac600/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:35.306115 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bad896b-f185-44f6-9481-aa77c9fb9d17 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:35.308334 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-0bad896b-f185-44f6-9481-aa77c9fb9d17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:35.323625 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bad896b-f185-44f6-9481-aa77c9fb9d17 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:35.326598 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42794580-2920-414d-99d1-d1e7caa455a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aada7626dfa44f5ebb843842c5859b51, user_id=f8b6d604fc594479a076d38a669ac600, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:35.329204 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cb2c5-9d88-423c-a991-49d2a1e60716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:35.329925 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cb2c5-9d88-423c-a991-49d2a1e60716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:35.330268 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cb2c5-9d88-423c-a991-49d2a1e60716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:35.330399 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42794580-2920-414d-99d1-d1e7caa455a9 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:35.330684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cb2c5-9d88-423c-a991-49d2a1e60716 admin admin] PATH_INFO: `/v3/projects/8c51cbf8c85c46b59db522cd25eebbd9` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:35.331928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303cb2c5-9d88-423c-a991-49d2a1e60716 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c51cbf8c85c46b59db522cd25eebbd9)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:35.335550 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303cb2c5-9d88-423c-a991-49d2a1e60716 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:35.367304 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 985/1966] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:35 2022] PUT /identity/v3/projects/aada7626dfa44f5ebb843842c5859b51/users/f8b6d604fc594479a076d38a669ac600/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:11:35.375207 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42794580-2920-414d-99d1-d1e7caa455a9 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:35.416842 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-303cb2c5-9d88-423c-a991-49d2a1e60716 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c51cbf8c85c46b59db522cd25eebbd9', 'request_id': 'req-303cb2c5-9d88-423c-a991-49d2a1e60716'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:35.422718 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-303cb2c5-9d88-423c-a991-49d2a1e60716 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c51cbf8c85c46b59db522cd25eebbd9', 'request_id': 'req-303cb2c5-9d88-423c-a991-49d2a1e60716'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:35.429277 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-303cb2c5-9d88-423c-a991-49d2a1e60716 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c51cbf8c85c46b59db522cd25eebbd9', 'request_id': 'req-303cb2c5-9d88-423c-a991-49d2a1e60716'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:35.442959 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-303cb2c5-9d88-423c-a991-49d2a1e60716 admin admin] The token cache is being invalidate because project 8c51cbf8c85c46b59db522cd25eebbd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:11:35.443384 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-303cb2c5-9d88-423c-a991-49d2a1e60716 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:35.444083 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-303cb2c5-9d88-423c-a991-49d2a1e60716 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:35.447110 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 982/1967] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:11:35 2022] DELETE /identity/v3/projects/8c51cbf8c85c46b59db522cd25eebbd9 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:35.447250 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42794580-2920-414d-99d1-d1e7caa455a9 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:35.450637 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd574f-cac3-4e8a-b1c3-6cf074bd37dd None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:35.450927 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd574f-cac3-4e8a-b1c3-6cf074bd37dd None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:35.451237 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd574f-cac3-4e8a-b1c3-6cf074bd37dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:35.473020 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-b4cd574f-cac3-4e8a-b1c3-6cf074bd37dd None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:35.474249 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-b4cd574f-cac3-4e8a-b1c3-6cf074bd37dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:35.488076 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:35.516397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cad08f-cfd4-476d-a929-18c43029019f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:35.517097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cad08f-cfd4-476d-a929-18c43029019f admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:35.517433 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cad08f-cfd4-476d-a929-18c43029019f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:35.517795 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cad08f-cfd4-476d-a929-18c43029019f admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:35.518501 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cad08f-cfd4-476d-a929-18c43029019f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:35.520226 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cad08f-cfd4-476d-a929-18c43029019f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:35.532574 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 986/1968] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:35.536559 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cad08f-cfd4-476d-a929-18c43029019f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:35.540973 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c0cad08f-cfd4-476d-a929-18c43029019f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:35.546566 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 983/1969] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:11:35.554553 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cad08f-cfd4-476d-a929-18c43029019f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:35.556789 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07aba50-7c50-4abb-8b43-1f769485042e None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:35.557259 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07aba50-7c50-4abb-8b43-1f769485042e None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:35.557538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07aba50-7c50-4abb-8b43-1f769485042e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:35.559951 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff1429-d58a-45ee-b32f-d8c6efc6ca03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:35.560637 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff1429-d58a-45ee-b32f-d8c6efc6ca03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:35.560889 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff1429-d58a-45ee-b32f-d8c6efc6ca03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:35.561164 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff1429-d58a-45ee-b32f-d8c6efc6ca03 admin admin] PATH_INFO: `/v3/projects/aada7626dfa44f5ebb843842c5859b51/users/f8b6d604fc594479a076d38a669ac600/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:35.563228 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff1429-d58a-45ee-b32f-d8c6efc6ca03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aada7626dfa44f5ebb843842c5859b51, user_id=f8b6d604fc594479a076d38a669ac600, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:35.571096 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff1429-d58a-45ee-b32f-d8c6efc6ca03 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:35.580179 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-b07aba50-7c50-4abb-8b43-1f769485042e None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:35.581381 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-b07aba50-7c50-4abb-8b43-1f769485042e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:35.584260 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 987/1970] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:35 2022] PUT /identity/v3/projects/aada7626dfa44f5ebb843842c5859b51/users/f8b6d604fc594479a076d38a669ac600/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:11:35.666881 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 984/1971] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:35.671737 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff1429-d58a-45ee-b32f-d8c6efc6ca03 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:35.685902 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff1429-d58a-45ee-b32f-d8c6efc6ca03 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:35.692648 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393cf5fa-a7ab-4406-bb6e-b77a03fc29ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:35.694027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cf5fa-a7ab-4406-bb6e-b77a03fc29ab admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:35.694308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cf5fa-a7ab-4406-bb6e-b77a03fc29ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:35.694577 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cf5fa-a7ab-4406-bb6e-b77a03fc29ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:35.695258 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393cf5fa-a7ab-4406-bb6e-b77a03fc29ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:11:35.695567 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393cf5fa-a7ab-4406-bb6e-b77a03fc29ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:35.697148 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393cf5fa-a7ab-4406-bb6e-b77a03fc29ab admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:35.713206 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 988/1972] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:11:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 13:11:35.714328 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07aba50-7c50-4abb-8b43-1f769485042e None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:35.724476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07aba50-7c50-4abb-8b43-1f769485042e None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:35.730419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef2172-bb75-40fc-bce6-6f15a2a9625a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:35.731148 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef2172-bb75-40fc-bce6-6f15a2a9625a admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:35.731519 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef2172-bb75-40fc-bce6-6f15a2a9625a admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:35.731858 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef2172-bb75-40fc-bce6-6f15a2a9625a admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:35.732711 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef2172-bb75-40fc-bce6-6f15a2a9625a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:35.734890 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef2172-bb75-40fc-bce6-6f15a2a9625a admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:35.767487 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 985/1973] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:11:35 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:35.771833 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393cf5fa-a7ab-4406-bb6e-b77a03fc29ab admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:35.784097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393cf5fa-a7ab-4406-bb6e-b77a03fc29ab admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:35.842367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:35.850661 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db321366-1ace-4cd1-96f1-a390b4a18f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:35.851407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db321366-1ace-4cd1-96f1-a390b4a18f5d admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:35.851652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db321366-1ace-4cd1-96f1-a390b4a18f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:35.852274 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db321366-1ace-4cd1-96f1-a390b4a18f5d admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:35.852788 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db321366-1ace-4cd1-96f1-a390b4a18f5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:35.855700 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db321366-1ace-4cd1-96f1-a390b4a18f5d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:35.883530 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 989/1974] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:11:35 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:11:35.884904 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef2172-bb75-40fc-bce6-6f15a2a9625a admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:35.895171 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef2172-bb75-40fc-bce6-6f15a2a9625a admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:35.899963 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72533d9-cf84-4dc6-b8b1-351aa97bc372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:35.900585 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72533d9-cf84-4dc6-b8b1-351aa97bc372 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:35.900847 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72533d9-cf84-4dc6-b8b1-351aa97bc372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:35.901116 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72533d9-cf84-4dc6-b8b1-351aa97bc372 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:35.901949 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72533d9-cf84-4dc6-b8b1-351aa97bc372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:35.903593 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72533d9-cf84-4dc6-b8b1-351aa97bc372 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:35.926204 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 986/1975] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:35.929302 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db321366-1ace-4cd1-96f1-a390b4a18f5d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:35.943119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db321366-1ace-4cd1-96f1-a390b4a18f5d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:35.948849 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125ce93-f27c-4154-aee4-8826ba2eb796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:35.949546 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125ce93-f27c-4154-aee4-8826ba2eb796 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:35.949896 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125ce93-f27c-4154-aee4-8826ba2eb796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:35.950359 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125ce93-f27c-4154-aee4-8826ba2eb796 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:35.951158 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c125ce93-f27c-4154-aee4-8826ba2eb796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:35.952767 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c125ce93-f27c-4154-aee4-8826ba2eb796 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:35.964747 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 990/1976] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:35.968716 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72533d9-cf84-4dc6-b8b1-351aa97bc372 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:35.979730 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72533d9-cf84-4dc6-b8b1-351aa97bc372 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:35.989631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bfcac-6fa3-473c-b7ab-61c1b9f2e243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:35.991030 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bfcac-6fa3-473c-b7ab-61c1b9f2e243 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:35.991363 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bfcac-6fa3-473c-b7ab-61c1b9f2e243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:35.991735 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bfcac-6fa3-473c-b7ab-61c1b9f2e243 admin admin] PATH_INFO: `/v3/projects/08ce700434064c0c955a12b546c1af85/users/c5c67b0f35334fabbb661361f9e30e2a/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:36.008668 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0bfcac-6fa3-473c-b7ab-61c1b9f2e243 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08ce700434064c0c955a12b546c1af85, user_id=c5c67b0f35334fabbb661361f9e30e2a, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:36.012608 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0bfcac-6fa3-473c-b7ab-61c1b9f2e243 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:36.027828 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 987/1977] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:35 2022] PUT /identity/v3/projects/08ce700434064c0c955a12b546c1af85/users/c5c67b0f35334fabbb661361f9e30e2a/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:36.038249 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e806d15-bf42-442e-ada6-28819c2346da None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:36.038527 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e806d15-bf42-442e-ada6-28819c2346da None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:36.038742 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e806d15-bf42-442e-ada6-28819c2346da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:36.076368 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-3e806d15-bf42-442e-ada6-28819c2346da None None] MFA Rules not processed for user `c5c67b0f35334fabbb661361f9e30e2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:36.077543 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3e806d15-bf42-442e-ada6-28819c2346da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:36.156592 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 991/1978] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:36.160742 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bfcac-6fa3-473c-b7ab-61c1b9f2e243 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:36.241866 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bfcac-6fa3-473c-b7ab-61c1b9f2e243 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:36.254980 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9ef1f-061c-43ca-8f93-c28c4a9bde1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:36.257001 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9ef1f-061c-43ca-8f93-c28c4a9bde1f admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:36.258318 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9ef1f-061c-43ca-8f93-c28c4a9bde1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:36.258440 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f9ef1f-061c-43ca-8f93-c28c4a9bde1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:36.260539 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9ef1f-061c-43ca-8f93-c28c4a9bde1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:36.265546 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f9ef1f-061c-43ca-8f93-c28c4a9bde1f admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:36.285436 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:36.322637 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 988/1979] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:11:36 2022] POST /identity/v3/projects => generated 383 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:36.326808 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e806d15-bf42-442e-ada6-28819c2346da None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:36.338822 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e806d15-bf42-442e-ada6-28819c2346da None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:36.424780 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5eb10-711c-43a9-9a90-fe7534028e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:36.425515 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5eb10-711c-43a9-9a90-fe7534028e83 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:36.425745 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5eb10-711c-43a9-9a90-fe7534028e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:36.426025 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5eb10-711c-43a9-9a90-fe7534028e83 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:36.426979 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5eb10-711c-43a9-9a90-fe7534028e83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:36.429381 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5eb10-711c-43a9-9a90-fe7534028e83 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:36.457040 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 992/1980] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:11:36 2022] POST /identity/v3/users => generated 292 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:36.461389 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9ef1f-061c-43ca-8f93-c28c4a9bde1f admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:36.479483 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f9ef1f-061c-43ca-8f93-c28c4a9bde1f admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:36.493543 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6bf53-d6f8-4cde-b2cc-a6970cccdce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:36.495629 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6bf53-d6f8-4cde-b2cc-a6970cccdce7 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:36.495956 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6bf53-d6f8-4cde-b2cc-a6970cccdce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:36.496243 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6bf53-d6f8-4cde-b2cc-a6970cccdce7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:36.497053 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6bf53-d6f8-4cde-b2cc-a6970cccdce7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:36.499267 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f6bf53-d6f8-4cde-b2cc-a6970cccdce7 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:36.529092 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 989/1981] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:36.529373 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5eb10-711c-43a9-9a90-fe7534028e83 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:36.540588 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5eb10-711c-43a9-9a90-fe7534028e83 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:36.549289 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d16ac-be42-43a5-8fdc-87193955440a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:36.550155 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d16ac-be42-43a5-8fdc-87193955440a admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:36.550572 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d16ac-be42-43a5-8fdc-87193955440a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:36.551217 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d16ac-be42-43a5-8fdc-87193955440a admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:36.552883 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d16ac-be42-43a5-8fdc-87193955440a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:36.555487 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d16ac-be42-43a5-8fdc-87193955440a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:36.573119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 993/1982] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:11:36.577236 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6bf53-d6f8-4cde-b2cc-a6970cccdce7 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:36.590378 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6bf53-d6f8-4cde-b2cc-a6970cccdce7 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:36.598287 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02428c70-5f98-4d36-8bc7-756d897e1dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:36.599525 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02428c70-5f98-4d36-8bc7-756d897e1dea admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:36.600273 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02428c70-5f98-4d36-8bc7-756d897e1dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:36.600816 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02428c70-5f98-4d36-8bc7-756d897e1dea admin admin] PATH_INFO: `/v3/projects/2021e7bac12f4658bb3fba5e31944e93/users/229d0131ac4445e0acfb2b18b98fb33a/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:36.639663 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02428c70-5f98-4d36-8bc7-756d897e1dea admin admin] RBAC: Authorizing `identity:create_grant(project_id=2021e7bac12f4658bb3fba5e31944e93, user_id=229d0131ac4445e0acfb2b18b98fb33a, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:36.648984 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02428c70-5f98-4d36-8bc7-756d897e1dea admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:36.679905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 990/1983] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:36 2022] PUT /identity/v3/projects/2021e7bac12f4658bb3fba5e31944e93/users/229d0131ac4445e0acfb2b18b98fb33a/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:36.690896 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631c309-341b-4b9b-8d99-d6201babdec0 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:36.691214 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631c309-341b-4b9b-8d99-d6201babdec0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:36.691494 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631c309-341b-4b9b-8d99-d6201babdec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:36.747288 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c631c309-341b-4b9b-8d99-d6201babdec0 None None] MFA Rules not processed for user `229d0131ac4445e0acfb2b18b98fb33a`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:36.748526 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c631c309-341b-4b9b-8d99-d6201babdec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:36.794458 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:36.835007 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2bc17-0c9a-493e-827f-5ae2016b618b None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:36.835323 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2bc17-0c9a-493e-827f-5ae2016b618b None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:36.835596 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2bc17-0c9a-493e-827f-5ae2016b618b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:36.861296 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 994/1984] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:36.867762 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631c309-341b-4b9b-8d99-d6201babdec0 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:36.874039 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-a3d2bc17-0c9a-493e-827f-5ae2016b618b None None] MFA Rules not processed for user `f8b6d604fc594479a076d38a669ac600`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:36.877314 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-a3d2bc17-0c9a-493e-827f-5ae2016b618b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:36.978360 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631c309-341b-4b9b-8d99-d6201babdec0 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:36.986203 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b1b1f-5016-4411-aa1f-7ceb2c50e888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:36.988818 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b1b1f-5016-4411-aa1f-7ceb2c50e888 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:36.990457 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b1b1f-5016-4411-aa1f-7ceb2c50e888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:36.992571 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b1b1f-5016-4411-aa1f-7ceb2c50e888 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:36.993698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b1b1f-5016-4411-aa1f-7ceb2c50e888 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:36.996535 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b1b1f-5016-4411-aa1f-7ceb2c50e888 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:37.024901 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 991/1985] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:36 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:11:37.039582 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2bc17-0c9a-493e-827f-5ae2016b618b None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:37.041126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-a3d2bc17-0c9a-493e-827f-5ae2016b618b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:37.054224 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 995/1986] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:11:36 2022] POST /identity/v3/projects => generated 385 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:37.060406 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b1b1f-5016-4411-aa1f-7ceb2c50e888 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:37.136310 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2bc17-0c9a-493e-827f-5ae2016b618b None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:37.143669 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b1b1f-5016-4411-aa1f-7ceb2c50e888 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:37.151570 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932631b-4e13-460a-8e81-0c557d5708ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:37.152174 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932631b-4e13-460a-8e81-0c557d5708ab admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:37.152450 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932631b-4e13-460a-8e81-0c557d5708ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:37.152718 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932631b-4e13-460a-8e81-0c557d5708ab admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:37.153348 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2932631b-4e13-460a-8e81-0c557d5708ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:37.155273 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2932631b-4e13-460a-8e81-0c557d5708ab admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:37.182435 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 996/1987] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:11:37 2022] POST /identity/v3/users => generated 299 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:11:37.183542 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932631b-4e13-460a-8e81-0c557d5708ab admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:37.189424 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4c298-bcaf-43e6-980b-46f2c8ced6db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:37.190653 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4c298-bcaf-43e6-980b-46f2c8ced6db service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:37.190938 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4c298-bcaf-43e6-980b-46f2c8ced6db service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:37.191403 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4c298-bcaf-43e6-980b-46f2c8ced6db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:37.193208 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-8ce4c298-bcaf-43e6-980b-46f2c8ced6db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:37.193582 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932631b-4e13-460a-8e81-0c557d5708ab admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:37.198421 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c26e9-9bc0-41c7-a9da-f032ab53426f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:37.199228 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c26e9-9bc0-41c7-a9da-f032ab53426f admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:37.199864 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c26e9-9bc0-41c7-a9da-f032ab53426f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:37.199864 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c26e9-9bc0-41c7-a9da-f032ab53426f admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:37.200922 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1c26e9-9bc0-41c7-a9da-f032ab53426f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:37.202521 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1c26e9-9bc0-41c7-a9da-f032ab53426f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:37.203632 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:37.213949 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 997/1988] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:11:37.219786 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c26e9-9bc0-41c7-a9da-f032ab53426f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:37.231607 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c26e9-9bc0-41c7-a9da-f032ab53426f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:37.236460 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0540e-da66-491f-9344-84c39900ae06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:37.237108 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0540e-da66-491f-9344-84c39900ae06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:37.237385 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0540e-da66-491f-9344-84c39900ae06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:37.237655 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0540e-da66-491f-9344-84c39900ae06 admin admin] PATH_INFO: `/v3/projects/6697e75a704d45b8b966474949e4544c/users/953dee526eef4fe19f227f8f514ad17c/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:37.256558 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be0540e-da66-491f-9344-84c39900ae06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6697e75a704d45b8b966474949e4544c, user_id=953dee526eef4fe19f227f8f514ad17c, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:37.256745 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce4c298-bcaf-43e6-980b-46f2c8ced6db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:37.258082 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce4c298-bcaf-43e6-980b-46f2c8ced6db service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:37.259603 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be0540e-da66-491f-9344-84c39900ae06 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:37.267761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 992/1989] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:11:37.274028 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 998/1990] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:37 2022] PUT /identity/v3/projects/6697e75a704d45b8b966474949e4544c/users/953dee526eef4fe19f227f8f514ad17c/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:37.277869 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4c298-bcaf-43e6-980b-46f2c8ced6db service cinder] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:37.288423 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4c298-bcaf-43e6-980b-46f2c8ced6db service cinder] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:37.345682 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d21de4-402a-48de-85b4-668cf0b49954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:37.346285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d21de4-402a-48de-85b4-668cf0b49954 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:37.346566 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d21de4-402a-48de-85b4-668cf0b49954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:37.346838 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d21de4-402a-48de-85b4-668cf0b49954 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:37.347571 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d21de4-402a-48de-85b4-668cf0b49954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:37.349253 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d21de4-402a-48de-85b4-668cf0b49954 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:37.362501 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 993/1991] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:37 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:37.363599 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0540e-da66-491f-9344-84c39900ae06 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:37.374595 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0540e-da66-491f-9344-84c39900ae06 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:37.379536 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999f225-cf8c-474e-a172-4fefe934b58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:37.380270 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999f225-cf8c-474e-a172-4fefe934b58c admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:37.381192 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999f225-cf8c-474e-a172-4fefe934b58c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:37.381466 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999f225-cf8c-474e-a172-4fefe934b58c admin admin] PATH_INFO: `/v3/projects/6697e75a704d45b8b966474949e4544c/users/953dee526eef4fe19f227f8f514ad17c/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:37.383577 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9999f225-cf8c-474e-a172-4fefe934b58c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6697e75a704d45b8b966474949e4544c, user_id=953dee526eef4fe19f227f8f514ad17c, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:37.386306 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9999f225-cf8c-474e-a172-4fefe934b58c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:37.393412 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:37.398582 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 999/1992] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:37 2022] PUT /identity/v3/projects/6697e75a704d45b8b966474949e4544c/users/953dee526eef4fe19f227f8f514ad17c/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:11:37.409824 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283cf9dc-a1a5-47e5-80b9-18e01e486f1e None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:37.410149 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283cf9dc-a1a5-47e5-80b9-18e01e486f1e None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:37.410407 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283cf9dc-a1a5-47e5-80b9-18e01e486f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:37.440224 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-283cf9dc-a1a5-47e5-80b9-18e01e486f1e None None] MFA Rules not processed for user `953dee526eef4fe19f227f8f514ad17c`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:37.441503 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-283cf9dc-a1a5-47e5-80b9-18e01e486f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:37.521705 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 994/1993] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:37 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:11:37.538339 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999f225-cf8c-474e-a172-4fefe934b58c admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:37.602386 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999f225-cf8c-474e-a172-4fefe934b58c admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:37.606065 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ca8db-5b01-4bb9-9f38-459e9d77ca4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:37.606673 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ca8db-5b01-4bb9-9f38-459e9d77ca4c service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:37.606976 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ca8db-5b01-4bb9-9f38-459e9d77ca4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:37.607242 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ca8db-5b01-4bb9-9f38-459e9d77ca4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:37.666456 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ca8db-5b01-4bb9-9f38-459e9d77ca4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:37.667975 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ca8db-5b01-4bb9-9f38-459e9d77ca4c service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:37.677804 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1000/1994] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:11:38.664043 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283cf9dc-a1a5-47e5-80b9-18e01e486f1e None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:38.665424 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-283cf9dc-a1a5-47e5-80b9-18e01e486f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:38.677291 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283cf9dc-a1a5-47e5-80b9-18e01e486f1e None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:38.680777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23bd65-ead5-43fa-8785-4c60be9ba0a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:38.681370 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23bd65-ead5-43fa-8785-4c60be9ba0a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:38.681648 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23bd65-ead5-43fa-8785-4c60be9ba0a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:38.681921 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23bd65-ead5-43fa-8785-4c60be9ba0a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:38.710438 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:38.741235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a23bd65-ead5-43fa-8785-4c60be9ba0a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:38.742664 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a23bd65-ead5-43fa-8785-4c60be9ba0a4 service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:38.752036 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 995/1995] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:11:39.216450 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ca8db-5b01-4bb9-9f38-459e9d77ca4c service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:39.225692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ca8db-5b01-4bb9-9f38-459e9d77ca4c service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:39.229457 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3351fac-c9b7-45e0-b789-191f03d2fa43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:39.230039 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3351fac-c9b7-45e0-b789-191f03d2fa43 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:39.230313 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3351fac-c9b7-45e0-b789-191f03d2fa43 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:39.230580 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3351fac-c9b7-45e0-b789-191f03d2fa43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:39.292472 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3351fac-c9b7-45e0-b789-191f03d2fa43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:39.294054 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3351fac-c9b7-45e0-b789-191f03d2fa43 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:39.304513 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1001/1996] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:11:39.841516 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23bd65-ead5-43fa-8785-4c60be9ba0a4 service cinder] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:39.849962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23bd65-ead5-43fa-8785-4c60be9ba0a4 service cinder] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:39.853532 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2be058a-6035-4509-b78a-1fcbda980ac6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:39.854123 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be058a-6035-4509-b78a-1fcbda980ac6 service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:39.854403 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be058a-6035-4509-b78a-1fcbda980ac6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:39.854673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be058a-6035-4509-b78a-1fcbda980ac6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:39.863396 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2be058a-6035-4509-b78a-1fcbda980ac6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:39.864808 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2be058a-6035-4509-b78a-1fcbda980ac6 service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:39.874011 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 996/1997] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:39 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:11:41.254832 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3351fac-c9b7-45e0-b789-191f03d2fa43 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:41.264373 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d3351fac-c9b7-45e0-b789-191f03d2fa43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:41.340440 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:41.355148 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3351fac-c9b7-45e0-b789-191f03d2fa43 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:41.359527 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee0cbd-b2b6-4040-ab2d-cfdaa3239618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:41.360154 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee0cbd-b2b6-4040-ab2d-cfdaa3239618 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:41.360434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee0cbd-b2b6-4040-ab2d-cfdaa3239618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:41.360706 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee0cbd-b2b6-4040-ab2d-cfdaa3239618 admin admin] PATH_INFO: `/v3/users/551300f824124d49b6e16bd932abe79a` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:41.361822 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee0cbd-b2b6-4040-ab2d-cfdaa3239618 admin admin] RBAC: Authorizing `identity:delete_user(user_id=551300f824124d49b6e16bd932abe79a)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:41.364055 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee0cbd-b2b6-4040-ab2d-cfdaa3239618 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:41.412227 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-e1ee0cbd-b2b6-4040-ab2d-cfdaa3239618 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '551300f824124d49b6e16bd932abe79a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:41.420737 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-e1ee0cbd-b2b6-4040-ab2d-cfdaa3239618 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '551300f824124d49b6e16bd932abe79a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:41.421537 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-e1ee0cbd-b2b6-4040-ab2d-cfdaa3239618 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '551300f824124d49b6e16bd932abe79a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:41.433059 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-e1ee0cbd-b2b6-4040-ab2d-cfdaa3239618 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '551300f824124d49b6e16bd932abe79a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:41.446444 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-e1ee0cbd-b2b6-4040-ab2d-cfdaa3239618 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '551300f824124d49b6e16bd932abe79a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:41.452355 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1002/1998] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:11:41 2022] DELETE /identity/v3/users/551300f824124d49b6e16bd932abe79a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:41.456886 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb56b0-57f5-42ca-bcf7-35e026743c3c None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:41.457369 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb56b0-57f5-42ca-bcf7-35e026743c3c None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:41.457536 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb56b0-57f5-42ca-bcf7-35e026743c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:41.489242 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-33fb56b0-57f5-42ca-bcf7-35e026743c3c None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:41.491236 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-33fb56b0-57f5-42ca-bcf7-35e026743c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:41.556823 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 997/1999] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:41.568722 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee0cbd-b2b6-4040-ab2d-cfdaa3239618 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:41.570191 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e1ee0cbd-b2b6-4040-ab2d-cfdaa3239618 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:41.638333 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee0cbd-b2b6-4040-ab2d-cfdaa3239618 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:41.642698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026fd836-e613-4eec-9737-aae91bdc72a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:41.643403 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fd836-e613-4eec-9737-aae91bdc72a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:41.643685 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fd836-e613-4eec-9737-aae91bdc72a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:41.643958 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fd836-e613-4eec-9737-aae91bdc72a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:41.652960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026fd836-e613-4eec-9737-aae91bdc72a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:41.654490 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026fd836-e613-4eec-9737-aae91bdc72a6 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:41.664020 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1003/2000] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:11:42.021833 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb56b0-57f5-42ca-bcf7-35e026743c3c None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:42.023272 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-33fb56b0-57f5-42ca-bcf7-35e026743c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:42.036930 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb56b0-57f5-42ca-bcf7-35e026743c3c None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:42.041462 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debc573-1fe7-47c0-a20a-726c6ee04f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:42.042086 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debc573-1fe7-47c0-a20a-726c6ee04f65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:42.042363 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debc573-1fe7-47c0-a20a-726c6ee04f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:42.042629 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debc573-1fe7-47c0-a20a-726c6ee04f65 admin admin] PATH_INFO: `/v3/projects/ff0bc2336bdd4b729f4b391e64e876b2` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:42.043757 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6debc573-1fe7-47c0-a20a-726c6ee04f65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff0bc2336bdd4b729f4b391e64e876b2)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:42.045661 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6debc573-1fe7-47c0-a20a-726c6ee04f65 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:42.094823 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-6debc573-1fe7-47c0-a20a-726c6ee04f65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff0bc2336bdd4b729f4b391e64e876b2', 'request_id': 'req-6debc573-1fe7-47c0-a20a-726c6ee04f65'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:42.100224 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-6debc573-1fe7-47c0-a20a-726c6ee04f65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff0bc2336bdd4b729f4b391e64e876b2', 'request_id': 'req-6debc573-1fe7-47c0-a20a-726c6ee04f65'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:42.106176 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-6debc573-1fe7-47c0-a20a-726c6ee04f65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff0bc2336bdd4b729f4b391e64e876b2', 'request_id': 'req-6debc573-1fe7-47c0-a20a-726c6ee04f65'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:42.118303 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-6debc573-1fe7-47c0-a20a-726c6ee04f65 admin admin] The token cache is being invalidate because project ff0bc2336bdd4b729f4b391e64e876b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:11:42.118607 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-6debc573-1fe7-47c0-a20a-726c6ee04f65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:42.119246 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-6debc573-1fe7-47c0-a20a-726c6ee04f65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:42.120726 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 998/2001] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:11:42 2022] DELETE /identity/v3/projects/ff0bc2336bdd4b729f4b391e64e876b2 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:42.125784 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb17b2-ecf2-4518-a408-b16d426e2388 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:42.126035 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb17b2-ecf2-4518-a408-b16d426e2388 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:42.126308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb17b2-ecf2-4518-a408-b16d426e2388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:42.145682 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-44fb17b2-ecf2-4518-a408-b16d426e2388 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:42.147474 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-44fb17b2-ecf2-4518-a408-b16d426e2388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:42.206216 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1004/2002] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:42.215634 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cfd54-5dbb-4203-9ff1-06eacc5cadb6 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:42.215923 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cfd54-5dbb-4203-9ff1-06eacc5cadb6 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:42.216194 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cfd54-5dbb-4203-9ff1-06eacc5cadb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:42.236007 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-e31cfd54-5dbb-4203-9ff1-06eacc5cadb6 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:42.237210 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-e31cfd54-5dbb-4203-9ff1-06eacc5cadb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:42.245014 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 999/2003] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:42.249229 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb17b2-ecf2-4518-a408-b16d426e2388 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:42.259047 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb17b2-ecf2-4518-a408-b16d426e2388 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:42.263654 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a40a0d-a455-4ae2-bcf9-df98241d6876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:42.264254 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a40a0d-a455-4ae2-bcf9-df98241d6876 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:42.264536 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a40a0d-a455-4ae2-bcf9-df98241d6876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:42.264810 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a40a0d-a455-4ae2-bcf9-df98241d6876 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:42.266795 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a40a0d-a455-4ae2-bcf9-df98241d6876 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:11:42.267166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a40a0d-a455-4ae2-bcf9-df98241d6876 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:42.268763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a40a0d-a455-4ae2-bcf9-df98241d6876 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:42.278303 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:42.282163 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1005/2004] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:11:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:11:42.287092 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31cfd54-5dbb-4203-9ff1-06eacc5cadb6 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:42.296423 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31cfd54-5dbb-4203-9ff1-06eacc5cadb6 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:42.301673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ad2a2-cd6f-4425-944a-cee2d4454913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:42.302387 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ad2a2-cd6f-4425-944a-cee2d4454913 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:42.302667 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ad2a2-cd6f-4425-944a-cee2d4454913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:42.302935 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ad2a2-cd6f-4425-944a-cee2d4454913 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:42.303638 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ad2a2-cd6f-4425-944a-cee2d4454913 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:42.305605 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ad2a2-cd6f-4425-944a-cee2d4454913 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:42.332132 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1000/2005] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:11:42 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:42.336684 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a40a0d-a455-4ae2-bcf9-df98241d6876 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:42.346317 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a40a0d-a455-4ae2-bcf9-df98241d6876 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:42.417941 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aecf57f-0d52-4588-8a45-0acaf6eac3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:42.418550 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aecf57f-0d52-4588-8a45-0acaf6eac3c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:42.418832 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aecf57f-0d52-4588-8a45-0acaf6eac3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:42.419133 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aecf57f-0d52-4588-8a45-0acaf6eac3c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:42.434003 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aecf57f-0d52-4588-8a45-0acaf6eac3c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:42.434003 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aecf57f-0d52-4588-8a45-0acaf6eac3c5 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:42.443220 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1006/2006] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:11:42 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:42.447086 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ad2a2-cd6f-4425-944a-cee2d4454913 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:42.457909 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ad2a2-cd6f-4425-944a-cee2d4454913 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:42.463133 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f736550-c37a-4df7-987f-1c4df3b8eaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:42.463732 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f736550-c37a-4df7-987f-1c4df3b8eaea admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:42.464013 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f736550-c37a-4df7-987f-1c4df3b8eaea admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:42.464283 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f736550-c37a-4df7-987f-1c4df3b8eaea admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:42.464921 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f736550-c37a-4df7-987f-1c4df3b8eaea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:42.466641 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f736550-c37a-4df7-987f-1c4df3b8eaea admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:42.467980 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:42.482758 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1001/2007] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:42.485372 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aecf57f-0d52-4588-8a45-0acaf6eac3c5 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:42.496878 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aecf57f-0d52-4588-8a45-0acaf6eac3c5 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:42.501936 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447ba7a-6516-424f-abab-844cb70b5c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:42.502541 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447ba7a-6516-424f-abab-844cb70b5c0b admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:42.502820 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447ba7a-6516-424f-abab-844cb70b5c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:42.503150 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447ba7a-6516-424f-abab-844cb70b5c0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:42.503769 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447ba7a-6516-424f-abab-844cb70b5c0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:42.506188 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447ba7a-6516-424f-abab-844cb70b5c0b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:42.517037 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1007/2008] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:42.521991 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f736550-c37a-4df7-987f-1c4df3b8eaea admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:42.531905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f736550-c37a-4df7-987f-1c4df3b8eaea admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:42.536761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc1050-a29a-44da-b5a4-10de22c7c0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:42.537426 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc1050-a29a-44da-b5a4-10de22c7c0fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:42.537823 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc1050-a29a-44da-b5a4-10de22c7c0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:42.538127 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc1050-a29a-44da-b5a4-10de22c7c0fe admin admin] PATH_INFO: `/v3/projects/e74136856dcf42c48689e6976f16caad/users/fd5dc5471e5246f4878f1d91d59b68f2/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:42.554341 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc1050-a29a-44da-b5a4-10de22c7c0fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=e74136856dcf42c48689e6976f16caad, user_id=fd5dc5471e5246f4878f1d91d59b68f2, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:42.557009 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc1050-a29a-44da-b5a4-10de22c7c0fe admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:42.570914 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1002/2009] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:42 2022] PUT /identity/v3/projects/e74136856dcf42c48689e6976f16caad/users/fd5dc5471e5246f4878f1d91d59b68f2/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:42.581157 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd280317-cd58-45aa-8052-2b6f40b5dae0 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:42.581447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd280317-cd58-45aa-8052-2b6f40b5dae0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:42.581722 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd280317-cd58-45aa-8052-2b6f40b5dae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:42.617305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-dd280317-cd58-45aa-8052-2b6f40b5dae0 None None] MFA Rules not processed for user `fd5dc5471e5246f4878f1d91d59b68f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:42.619178 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-dd280317-cd58-45aa-8052-2b6f40b5dae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:42.700281 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1008/2010] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:42.711595 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc1050-a29a-44da-b5a4-10de22c7c0fe admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:42.713418 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-c6dc1050-a29a-44da-b5a4-10de22c7c0fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:42.771670 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc1050-a29a-44da-b5a4-10de22c7c0fe admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:42.776167 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a031b-0245-4896-b24d-2605a8ee6869 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:42.777051 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a031b-0245-4896-b24d-2605a8ee6869 service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:42.777375 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a031b-0245-4896-b24d-2605a8ee6869 service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:42.777700 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a031b-0245-4896-b24d-2605a8ee6869 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:42.787156 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176a031b-0245-4896-b24d-2605a8ee6869 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:42.789099 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176a031b-0245-4896-b24d-2605a8ee6869 service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:42.797467 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1003/2011] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:11:43.771901 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd280317-cd58-45aa-8052-2b6f40b5dae0 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:43.773614 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-dd280317-cd58-45aa-8052-2b6f40b5dae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:43.844644 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd280317-cd58-45aa-8052-2b6f40b5dae0 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:43.849581 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09764eaf-4a41-4acd-b187-55a8fb867613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:43.850196 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09764eaf-4a41-4acd-b187-55a8fb867613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:43.850480 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09764eaf-4a41-4acd-b187-55a8fb867613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:43.850751 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09764eaf-4a41-4acd-b187-55a8fb867613 admin admin] PATH_INFO: `/v3/users/c5c67b0f35334fabbb661361f9e30e2a` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:43.851944 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09764eaf-4a41-4acd-b187-55a8fb867613 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5c67b0f35334fabbb661361f9e30e2a)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:43.853889 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09764eaf-4a41-4acd-b187-55a8fb867613 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:43.866113 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:43.929077 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-09764eaf-4a41-4acd-b187-55a8fb867613 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5c67b0f35334fabbb661361f9e30e2a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:43.938581 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-09764eaf-4a41-4acd-b187-55a8fb867613 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5c67b0f35334fabbb661361f9e30e2a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:43.939329 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-09764eaf-4a41-4acd-b187-55a8fb867613 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5c67b0f35334fabbb661361f9e30e2a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:43.955517 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-09764eaf-4a41-4acd-b187-55a8fb867613 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5c67b0f35334fabbb661361f9e30e2a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:43.961508 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-09764eaf-4a41-4acd-b187-55a8fb867613 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5c67b0f35334fabbb661361f9e30e2a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:43.968727 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1009/2012] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:11:43 2022] DELETE /identity/v3/users/c5c67b0f35334fabbb661361f9e30e2a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:43.977446 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a3f05-e525-4c0f-a03b-716030534584 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:43.977951 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a3f05-e525-4c0f-a03b-716030534584 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:43.978477 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a3f05-e525-4c0f-a03b-716030534584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:43.999807 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-763a3f05-e525-4c0f-a03b-716030534584 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:44.001005 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-763a3f05-e525-4c0f-a03b-716030534584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:44.066105 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1004/2013] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:44.076142 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09764eaf-4a41-4acd-b187-55a8fb867613 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:44.077519 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-09764eaf-4a41-4acd-b187-55a8fb867613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:44.137509 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09764eaf-4a41-4acd-b187-55a8fb867613 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:44.142892 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a8a3a-90aa-4172-b643-b71ea347d26c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:44.143635 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a8a3a-90aa-4172-b643-b71ea347d26c service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:44.143958 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a8a3a-90aa-4172-b643-b71ea347d26c service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:44.144308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a8a3a-90aa-4172-b643-b71ea347d26c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:44.156040 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5a8a3a-90aa-4172-b643-b71ea347d26c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:44.157536 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5a8a3a-90aa-4172-b643-b71ea347d26c service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:44.169470 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1010/2014] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:11:44.219027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ffef1f-99bf-415d-afdb-d1dffc462fb0 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:44.219347 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ffef1f-99bf-415d-afdb-d1dffc462fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:44.219672 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ffef1f-99bf-415d-afdb-d1dffc462fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:44.251262 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-51ffef1f-99bf-415d-afdb-d1dffc462fb0 None None] MFA Rules not processed for user `fd5dc5471e5246f4878f1d91d59b68f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:44.251262 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-51ffef1f-99bf-415d-afdb-d1dffc462fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:44.323965 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1005/2015] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:44.708238 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a8a3a-90aa-4172-b643-b71ea347d26c service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:44.709960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5c5a8a3a-90aa-4172-b643-b71ea347d26c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:44.761787 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a8a3a-90aa-4172-b643-b71ea347d26c service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:44.763102 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795d3e8-8a89-4de9-b1ef-0139414d7613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:44.764439 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795d3e8-8a89-4de9-b1ef-0139414d7613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:44.764807 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795d3e8-8a89-4de9-b1ef-0139414d7613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:44.765148 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795d3e8-8a89-4de9-b1ef-0139414d7613 admin admin] PATH_INFO: `/v3/projects/08ce700434064c0c955a12b546c1af85` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:44.766786 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8795d3e8-8a89-4de9-b1ef-0139414d7613 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08ce700434064c0c955a12b546c1af85)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:44.769579 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8795d3e8-8a89-4de9-b1ef-0139414d7613 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:44.840719 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-8795d3e8-8a89-4de9-b1ef-0139414d7613 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08ce700434064c0c955a12b546c1af85', 'request_id': 'req-8795d3e8-8a89-4de9-b1ef-0139414d7613'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:44.847022 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-8795d3e8-8a89-4de9-b1ef-0139414d7613 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08ce700434064c0c955a12b546c1af85', 'request_id': 'req-8795d3e8-8a89-4de9-b1ef-0139414d7613'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:44.854852 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-8795d3e8-8a89-4de9-b1ef-0139414d7613 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08ce700434064c0c955a12b546c1af85', 'request_id': 'req-8795d3e8-8a89-4de9-b1ef-0139414d7613'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:44.876916 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-8795d3e8-8a89-4de9-b1ef-0139414d7613 admin admin] The token cache is being invalidate because project 08ce700434064c0c955a12b546c1af85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:11:44.878358 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-8795d3e8-8a89-4de9-b1ef-0139414d7613 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:44.879534 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-8795d3e8-8a89-4de9-b1ef-0139414d7613 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:44.881546 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1011/2016] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:11:44 2022] DELETE /identity/v3/projects/08ce700434064c0c955a12b546c1af85 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:44.886282 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ffef1f-99bf-415d-afdb-d1dffc462fb0 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:44.898134 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ffef1f-99bf-415d-afdb-d1dffc462fb0 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:44.926895 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:44.981017 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64290864-3fea-4a39-872f-f267eac28232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:44.981796 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64290864-3fea-4a39-872f-f267eac28232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:44.982217 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64290864-3fea-4a39-872f-f267eac28232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:44.982498 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64290864-3fea-4a39-872f-f267eac28232 admin admin] PATH_INFO: `/v3/users/229d0131ac4445e0acfb2b18b98fb33a` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:44.983951 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64290864-3fea-4a39-872f-f267eac28232 admin admin] RBAC: Authorizing `identity:delete_user(user_id=229d0131ac4445e0acfb2b18b98fb33a)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:44.986494 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64290864-3fea-4a39-872f-f267eac28232 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:45.077430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-64290864-3fea-4a39-872f-f267eac28232 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '229d0131ac4445e0acfb2b18b98fb33a'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:45.088201 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-64290864-3fea-4a39-872f-f267eac28232 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '229d0131ac4445e0acfb2b18b98fb33a'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:45.089192 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-64290864-3fea-4a39-872f-f267eac28232 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '229d0131ac4445e0acfb2b18b98fb33a'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:45.090112 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-64290864-3fea-4a39-872f-f267eac28232 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '229d0131ac4445e0acfb2b18b98fb33a'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:45.101907 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-64290864-3fea-4a39-872f-f267eac28232 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '229d0131ac4445e0acfb2b18b98fb33a'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:45.122965 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1006/2017] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:11:44 2022] DELETE /identity/v3/users/229d0131ac4445e0acfb2b18b98fb33a => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:45.499322 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795d3e8-8a89-4de9-b1ef-0139414d7613 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:45.501842 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8795d3e8-8a89-4de9-b1ef-0139414d7613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:45.529349 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795d3e8-8a89-4de9-b1ef-0139414d7613 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:45.541140 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:45.608713 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928cd27-b9b2-4c33-b34d-1ab91ad3bb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:45.609359 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928cd27-b9b2-4c33-b34d-1ab91ad3bb73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:45.609747 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928cd27-b9b2-4c33-b34d-1ab91ad3bb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:45.610021 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928cd27-b9b2-4c33-b34d-1ab91ad3bb73 admin admin] PATH_INFO: `/v3/projects/2021e7bac12f4658bb3fba5e31944e93` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:45.611428 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928cd27-b9b2-4c33-b34d-1ab91ad3bb73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2021e7bac12f4658bb3fba5e31944e93)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:45.614161 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928cd27-b9b2-4c33-b34d-1ab91ad3bb73 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:45.670184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c928cd27-b9b2-4c33-b34d-1ab91ad3bb73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2021e7bac12f4658bb3fba5e31944e93', 'request_id': 'req-c928cd27-b9b2-4c33-b34d-1ab91ad3bb73'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:45.675797 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c928cd27-b9b2-4c33-b34d-1ab91ad3bb73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2021e7bac12f4658bb3fba5e31944e93', 'request_id': 'req-c928cd27-b9b2-4c33-b34d-1ab91ad3bb73'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:45.682692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c928cd27-b9b2-4c33-b34d-1ab91ad3bb73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2021e7bac12f4658bb3fba5e31944e93', 'request_id': 'req-c928cd27-b9b2-4c33-b34d-1ab91ad3bb73'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:45.694673 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c928cd27-b9b2-4c33-b34d-1ab91ad3bb73 admin admin] The token cache is being invalidate because project 2021e7bac12f4658bb3fba5e31944e93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:11:45.694958 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c928cd27-b9b2-4c33-b34d-1ab91ad3bb73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:45.695595 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c928cd27-b9b2-4c33-b34d-1ab91ad3bb73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:45.697073 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1012/2018] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:11:45 2022] DELETE /identity/v3/projects/2021e7bac12f4658bb3fba5e31944e93 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:45.701039 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64290864-3fea-4a39-872f-f267eac28232 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:45.702440 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-64290864-3fea-4a39-872f-f267eac28232 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:45.714938 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64290864-3fea-4a39-872f-f267eac28232 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:45.769906 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048736d-8089-4343-84ad-950b1f1af8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:45.770457 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048736d-8089-4343-84ad-950b1f1af8aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:45.770766 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048736d-8089-4343-84ad-950b1f1af8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:45.771041 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048736d-8089-4343-84ad-950b1f1af8aa admin admin] PATH_INFO: `/v3/users/953dee526eef4fe19f227f8f514ad17c` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:45.772247 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d048736d-8089-4343-84ad-950b1f1af8aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=953dee526eef4fe19f227f8f514ad17c)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:45.774161 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d048736d-8089-4343-84ad-950b1f1af8aa admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:45.815339 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:45.822994 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-d048736d-8089-4343-84ad-950b1f1af8aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '953dee526eef4fe19f227f8f514ad17c'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:45.831468 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-d048736d-8089-4343-84ad-950b1f1af8aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '953dee526eef4fe19f227f8f514ad17c'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:45.832320 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-d048736d-8089-4343-84ad-950b1f1af8aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '953dee526eef4fe19f227f8f514ad17c'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:45.832918 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-d048736d-8089-4343-84ad-950b1f1af8aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '953dee526eef4fe19f227f8f514ad17c'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:45.840989 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-d048736d-8089-4343-84ad-950b1f1af8aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '953dee526eef4fe19f227f8f514ad17c'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:45.858589 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1007/2019] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:11:45 2022] DELETE /identity/v3/users/953dee526eef4fe19f227f8f514ad17c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:46.155451 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928cd27-b9b2-4c33-b34d-1ab91ad3bb73 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:46.157044 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c928cd27-b9b2-4c33-b34d-1ab91ad3bb73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:46.172014 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928cd27-b9b2-4c33-b34d-1ab91ad3bb73 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:46.229354 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc54bcc-858a-4a5b-b909-3fcb63e0d624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:46.229976 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc54bcc-858a-4a5b-b909-3fcb63e0d624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:46.230253 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc54bcc-858a-4a5b-b909-3fcb63e0d624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:46.230522 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc54bcc-858a-4a5b-b909-3fcb63e0d624 admin admin] PATH_INFO: `/v3/projects/6697e75a704d45b8b966474949e4544c` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:46.231799 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc54bcc-858a-4a5b-b909-3fcb63e0d624 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6697e75a704d45b8b966474949e4544c)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:46.233691 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc54bcc-858a-4a5b-b909-3fcb63e0d624 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:46.279829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:46.283999 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3bc54bcc-858a-4a5b-b909-3fcb63e0d624 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6697e75a704d45b8b966474949e4544c', 'request_id': 'req-3bc54bcc-858a-4a5b-b909-3fcb63e0d624'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:46.299229 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3bc54bcc-858a-4a5b-b909-3fcb63e0d624 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6697e75a704d45b8b966474949e4544c', 'request_id': 'req-3bc54bcc-858a-4a5b-b909-3fcb63e0d624'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:46.299229 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3bc54bcc-858a-4a5b-b909-3fcb63e0d624 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6697e75a704d45b8b966474949e4544c', 'request_id': 'req-3bc54bcc-858a-4a5b-b909-3fcb63e0d624'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:46.313330 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3bc54bcc-858a-4a5b-b909-3fcb63e0d624 admin admin] The token cache is being invalidate because project 6697e75a704d45b8b966474949e4544c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:11:46.313554 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3bc54bcc-858a-4a5b-b909-3fcb63e0d624 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:46.314178 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3bc54bcc-858a-4a5b-b909-3fcb63e0d624 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:11:46.315766 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1013/2020] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:11:46 2022] DELETE /identity/v3/projects/6697e75a704d45b8b966474949e4544c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:46.321175 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30454ac1-8a2f-4bff-aec0-0f630a53cf5e None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:46.321657 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30454ac1-8a2f-4bff-aec0-0f630a53cf5e None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:46.321788 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30454ac1-8a2f-4bff-aec0-0f630a53cf5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:46.341460 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-30454ac1-8a2f-4bff-aec0-0f630a53cf5e None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:46.342637 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-30454ac1-8a2f-4bff-aec0-0f630a53cf5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:46.400678 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1008/2021] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:46.408594 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240ecad-5217-4611-ac3a-9f69004a9048 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:46.408878 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240ecad-5217-4611-ac3a-9f69004a9048 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:46.409156 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240ecad-5217-4611-ac3a-9f69004a9048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:46.428461 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-7240ecad-5217-4611-ac3a-9f69004a9048 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:46.429636 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7240ecad-5217-4611-ac3a-9f69004a9048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:46.437399 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1014/2022] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:11:46.441481 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30454ac1-8a2f-4bff-aec0-0f630a53cf5e None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:46.451580 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30454ac1-8a2f-4bff-aec0-0f630a53cf5e None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:46.455996 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca9c22-1292-482c-9def-e3fc273c30de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:46.456584 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca9c22-1292-482c-9def-e3fc273c30de admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:46.456860 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca9c22-1292-482c-9def-e3fc273c30de admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:46.457130 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca9c22-1292-482c-9def-e3fc273c30de admin admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:46.457764 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca9c22-1292-482c-9def-e3fc273c30de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:11:46.458060 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca9c22-1292-482c-9def-e3fc273c30de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:46.460236 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca9c22-1292-482c-9def-e3fc273c30de admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:46.471605 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1009/2023] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:11:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 13:11:46.475657 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240ecad-5217-4611-ac3a-9f69004a9048 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:46.490154 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240ecad-5217-4611-ac3a-9f69004a9048 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:46.494777 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f106430-be01-42cc-9fe1-5d2715ef9d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:46.495413 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f106430-be01-42cc-9fe1-5d2715ef9d5d admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:46.495694 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f106430-be01-42cc-9fe1-5d2715ef9d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:46.495965 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f106430-be01-42cc-9fe1-5d2715ef9d5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:46.496604 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f106430-be01-42cc-9fe1-5d2715ef9d5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:46.498487 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f106430-be01-42cc-9fe1-5d2715ef9d5d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:46.526305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1015/2024] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:11:46 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:46.530346 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca9c22-1292-482c-9def-e3fc273c30de admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:46.540180 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca9c22-1292-482c-9def-e3fc273c30de admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:46.596641 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98094505-7da8-4a91-a0d0-742dcef2a5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:46.597244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98094505-7da8-4a91-a0d0-742dcef2a5cf admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:46.597539 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98094505-7da8-4a91-a0d0-742dcef2a5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:46.597799 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98094505-7da8-4a91-a0d0-742dcef2a5cf admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:46.598440 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98094505-7da8-4a91-a0d0-742dcef2a5cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:46.600419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98094505-7da8-4a91-a0d0-742dcef2a5cf admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:46.622227 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1010/2025] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:11:46 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:46.626338 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f106430-be01-42cc-9fe1-5d2715ef9d5d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:46.637564 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f106430-be01-42cc-9fe1-5d2715ef9d5d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:46.644101 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481cab0f-0530-4358-a374-0ccdf1b8c8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:46.644762 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481cab0f-0530-4358-a374-0ccdf1b8c8b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:46.644946 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481cab0f-0530-4358-a374-0ccdf1b8c8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:46.645307 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481cab0f-0530-4358-a374-0ccdf1b8c8b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:46.646582 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481cab0f-0530-4358-a374-0ccdf1b8c8b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:46.648189 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481cab0f-0530-4358-a374-0ccdf1b8c8b6 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:46.658975 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1016/2026] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:11:46.662855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98094505-7da8-4a91-a0d0-742dcef2a5cf admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:46.673332 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98094505-7da8-4a91-a0d0-742dcef2a5cf admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:46.678791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8eb04a-3b30-45c3-9697-dfb73eb96d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:46.679651 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8eb04a-3b30-45c3-9697-dfb73eb96d20 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:46.679897 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8eb04a-3b30-45c3-9697-dfb73eb96d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:46.680174 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8eb04a-3b30-45c3-9697-dfb73eb96d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:46.680826 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8eb04a-3b30-45c3-9697-dfb73eb96d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:46.682465 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8eb04a-3b30-45c3-9697-dfb73eb96d20 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:46.694336 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1011/2027] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:46.698353 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481cab0f-0530-4358-a374-0ccdf1b8c8b6 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:46.708424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481cab0f-0530-4358-a374-0ccdf1b8c8b6 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:46.712979 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f491ba3-7c25-4f15-870e-0aae48702dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:46.713756 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f491ba3-7c25-4f15-870e-0aae48702dca admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:46.713988 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f491ba3-7c25-4f15-870e-0aae48702dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:46.714259 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f491ba3-7c25-4f15-870e-0aae48702dca admin admin] PATH_INFO: `/v3/projects/3ed5038c799342d48edc980983f1d996/users/7d89667e52b24295ba619203c02c7bd4/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:46.729848 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f491ba3-7c25-4f15-870e-0aae48702dca admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ed5038c799342d48edc980983f1d996, user_id=7d89667e52b24295ba619203c02c7bd4, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:46.732520 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f491ba3-7c25-4f15-870e-0aae48702dca admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:46.746804 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1017/2028] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:46 2022] PUT /identity/v3/projects/3ed5038c799342d48edc980983f1d996/users/7d89667e52b24295ba619203c02c7bd4/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:46.752037 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37596e51-ad63-44f4-b5bf-bdfa2b794c07 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:46.752324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37596e51-ad63-44f4-b5bf-bdfa2b794c07 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:46.752597 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37596e51-ad63-44f4-b5bf-bdfa2b794c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:46.771596 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-37596e51-ad63-44f4-b5bf-bdfa2b794c07 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:46.772967 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-37596e51-ad63-44f4-b5bf-bdfa2b794c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:46.790788 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:46.835239 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1012/2029] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:46.844850 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f491ba3-7c25-4f15-870e-0aae48702dca admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:46.846534 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9f491ba3-7c25-4f15-870e-0aae48702dca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:46.903262 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f491ba3-7c25-4f15-870e-0aae48702dca admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:46.906922 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29c587-bdc5-4744-8b9c-ff33cdcc1978 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:46.907546 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29c587-bdc5-4744-8b9c-ff33cdcc1978 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:46.907826 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29c587-bdc5-4744-8b9c-ff33cdcc1978 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:46.908093 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c29c587-bdc5-4744-8b9c-ff33cdcc1978 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:46.917183 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c29c587-bdc5-4744-8b9c-ff33cdcc1978 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:46.918600 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c29c587-bdc5-4744-8b9c-ff33cdcc1978 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:46.928789 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1018/2030] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:11:52.643543 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa457dc-12b9-4c97-9683-c9200de24f9a None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:52.644122 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa457dc-12b9-4c97-9683-c9200de24f9a None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:52.644122 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa457dc-12b9-4c97-9683-c9200de24f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:52.690514 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-7fa457dc-12b9-4c97-9683-c9200de24f9a None None] MFA Rules not processed for user `7d89667e52b24295ba619203c02c7bd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:52.691914 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-7fa457dc-12b9-4c97-9683-c9200de24f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:52.779120 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1013/2031] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:52.781500 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29c587-bdc5-4744-8b9c-ff33cdcc1978 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:52.794646 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c29c587-bdc5-4744-8b9c-ff33cdcc1978 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:52.800179 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ddda5-e128-4e93-b036-f376b5568e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:52.800792 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ddda5-e128-4e93-b036-f376b5568e5a admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:52.801071 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ddda5-e128-4e93-b036-f376b5568e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:52.801628 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ddda5-e128-4e93-b036-f376b5568e5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:52.802271 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ddda5-e128-4e93-b036-f376b5568e5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:52.804222 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ddda5-e128-4e93-b036-f376b5568e5a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:52.844615 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1019/2032] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:11:52 2022] POST /identity/v3/projects => generated 383 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:52.849276 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa457dc-12b9-4c97-9683-c9200de24f9a None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:52.860359 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa457dc-12b9-4c97-9683-c9200de24f9a None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:52.938119 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fda8c8-069e-4528-b496-30f03c0c9b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:52.939023 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fda8c8-069e-4528-b496-30f03c0c9b46 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:52.939284 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fda8c8-069e-4528-b496-30f03c0c9b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:52.939567 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fda8c8-069e-4528-b496-30f03c0c9b46 admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:52.941236 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fda8c8-069e-4528-b496-30f03c0c9b46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:52.944353 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fda8c8-069e-4528-b496-30f03c0c9b46 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:52.979097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ddda5-e128-4e93-b036-f376b5568e5a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:52.994690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1014/2033] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:11:52 2022] POST /identity/v3/users => generated 298 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:11:53.004330 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ddda5-e128-4e93-b036-f376b5568e5a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:53.011436 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4ee2ea-535d-4a06-961c-a1bcd509f4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:53.012186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ee2ea-535d-4a06-961c-a1bcd509f4e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:53.012582 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ee2ea-535d-4a06-961c-a1bcd509f4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:53.013544 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ee2ea-535d-4a06-961c-a1bcd509f4e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:53.014380 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4ee2ea-535d-4a06-961c-a1bcd509f4e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:53.016177 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4ee2ea-535d-4a06-961c-a1bcd509f4e0 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:53.036500 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fda8c8-069e-4528-b496-30f03c0c9b46 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:53.049338 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1020/2034] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:52 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:11:53.050695 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fda8c8-069e-4528-b496-30f03c0c9b46 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:53.057398 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb284e-aa20-49f2-9ca9-87140d603ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:53.058112 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb284e-aa20-49f2-9ca9-87140d603ddc admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:53.058487 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb284e-aa20-49f2-9ca9-87140d603ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:53.058839 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb284e-aa20-49f2-9ca9-87140d603ddc admin admin] PATH_INFO: `/v3/projects/e5527143235a4fab8d08d7ef484670ce/users/afa565808a2d4ea9b6274cabcf1f4572/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:53.075466 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eb284e-aa20-49f2-9ca9-87140d603ddc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5527143235a4fab8d08d7ef484670ce, user_id=afa565808a2d4ea9b6274cabcf1f4572, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:53.078276 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eb284e-aa20-49f2-9ca9-87140d603ddc admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:53.094117 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1015/2035] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:53 2022] PUT /identity/v3/projects/e5527143235a4fab8d08d7ef484670ce/users/afa565808a2d4ea9b6274cabcf1f4572/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:11:53.097956 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4ee2ea-535d-4a06-961c-a1bcd509f4e0 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:53.109317 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4ee2ea-535d-4a06-961c-a1bcd509f4e0 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:53.157980 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:53.188479 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff237855-c4bf-4d80-ba23-672c0d6100d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:53.189371 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff237855-c4bf-4d80-ba23-672c0d6100d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:53.191000 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff237855-c4bf-4d80-ba23-672c0d6100d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:53.193723 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff237855-c4bf-4d80-ba23-672c0d6100d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:53.195736 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff237855-c4bf-4d80-ba23-672c0d6100d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:53.197649 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff237855-c4bf-4d80-ba23-672c0d6100d8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:53.213936 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1021/2036] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:11:53.215173 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb284e-aa20-49f2-9ca9-87140d603ddc admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:53.228347 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb284e-aa20-49f2-9ca9-87140d603ddc admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:53.233870 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef72d762-4d60-433c-8533-e23987b42e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:53.234542 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72d762-4d60-433c-8533-e23987b42e40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:53.234824 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72d762-4d60-433c-8533-e23987b42e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:53.235131 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72d762-4d60-433c-8533-e23987b42e40 admin admin] PATH_INFO: `/v3/projects/e5527143235a4fab8d08d7ef484670ce/users/afa565808a2d4ea9b6274cabcf1f4572/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:53.237120 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef72d762-4d60-433c-8533-e23987b42e40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5527143235a4fab8d08d7ef484670ce, user_id=afa565808a2d4ea9b6274cabcf1f4572, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:53.240163 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef72d762-4d60-433c-8533-e23987b42e40 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:53.252109 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1016/2037] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:11:53 2022] PUT /identity/v3/projects/e5527143235a4fab8d08d7ef484670ce/users/afa565808a2d4ea9b6274cabcf1f4572/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:11:58.699689 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4128b98-e1d5-471b-b155-c3d3109137d6 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:58.700238 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4128b98-e1d5-471b-b155-c3d3109137d6 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:58.700238 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4128b98-e1d5-471b-b155-c3d3109137d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:58.738703 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f4128b98-e1d5-471b-b155-c3d3109137d6 None None] MFA Rules not processed for user `afa565808a2d4ea9b6274cabcf1f4572`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:58.740120 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f4128b98-e1d5-471b-b155-c3d3109137d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:58.814356 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1022/2038] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:11:58 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:11:58.826659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef72d762-4d60-433c-8533-e23987b42e40 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:58.831734 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-ef72d762-4d60-433c-8533-e23987b42e40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:58.849917 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:11:58.926086 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef72d762-4d60-433c-8533-e23987b42e40 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:58.936825 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93388afc-5d83-48dc-ace8-98582f167439 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ce234c5b4643b2ac6341e0928f3830', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:58.939367 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93388afc-5d83-48dc-ace8-98582f167439 service glance] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:58.939715 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93388afc-5d83-48dc-ace8-98582f167439 service glance] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:58.940062 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93388afc-5d83-48dc-ace8-98582f167439 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:59.097495 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93388afc-5d83-48dc-ace8-98582f167439 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:59.099449 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93388afc-5d83-48dc-ace8-98582f167439 service glance] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:59.122982 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1017/2039] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:11:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:11:59.362929 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29bf1c-dba8-4f95-98bc-ee0e11cad930 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:59.363797 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29bf1c-dba8-4f95-98bc-ee0e11cad930 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:59.365225 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29bf1c-dba8-4f95-98bc-ee0e11cad930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:59.392451 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6c29bf1c-dba8-4f95-98bc-ee0e11cad930 None None] MFA Rules not processed for user `b6ae674f0efc445fa970fca64e13893a`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:11:59.394196 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6c29bf1c-dba8-4f95-98bc-ee0e11cad930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:59.546869 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1023/2040] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Oct 13 13:11:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:11:59.615146 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93388afc-5d83-48dc-ace8-98582f167439 service glance] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:11:59.616557 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-93388afc-5d83-48dc-ace8-98582f167439 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:11:59.782301 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93388afc-5d83-48dc-ace8-98582f167439 service glance] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:11:59.793000 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd8204f-03f8-4e96-aeec-81514d833372 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526000341fd146db8c661f1fd14d4801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:11:59.793786 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd8204f-03f8-4e96-aeec-81514d833372 service swift] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:11:59.794267 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd8204f-03f8-4e96-aeec-81514d833372 service swift] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:11:59.794606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd8204f-03f8-4e96-aeec-81514d833372 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:11:59.805091 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd8204f-03f8-4e96-aeec-81514d833372 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:11:59.806976 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd8204f-03f8-4e96-aeec-81514d833372 service swift] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:11:59.823367 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1018/2041] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 13 13:11:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 13 13:12:00.678917 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35894d-1586-4d91-a0fc-8834d54b8a13 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:00.679551 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35894d-1586-4d91-a0fc-8834d54b8a13 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:00.679551 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35894d-1586-4d91-a0fc-8834d54b8a13 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:00.680959 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1024/2042] 10.222.0.23 () {54 vars in 921 bytes} [Thu Oct 13 13:12:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:12:00.686339 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8cefd5-07b6-41a1-9126-04211344f58b None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:00.686883 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8cefd5-07b6-41a1-9126-04211344f58b None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:00.686883 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8cefd5-07b6-41a1-9126-04211344f58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:00.721542 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-6f8cefd5-07b6-41a1-9126-04211344f58b None None] MFA Rules not processed for user `1a971cdb7f9d4e6692c6326c103d7303`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:12:00.722779 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-6f8cefd5-07b6-41a1-9126-04211344f58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:00.920320 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:12:00.970152 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1019/2043] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Oct 13 13:12:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:12:01.009688 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35894d-1586-4d91-a0fc-8834d54b8a13 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:01.012122 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8c35894d-1586-4d91-a0fc-8834d54b8a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:01.055823 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35894d-1586-4d91-a0fc-8834d54b8a13 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:01.071142 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654f2d8b-8a06-488c-b78b-23548abba132 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a971cdb7f9d4e6692c6326c103d7303', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:01.071936 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f2d8b-8a06-488c-b78b-23548abba132 service placement] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:01.072649 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f2d8b-8a06-488c-b78b-23548abba132 service placement] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:01.072649 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f2d8b-8a06-488c-b78b-23548abba132 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:01.095852 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654f2d8b-8a06-488c-b78b-23548abba132 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:01.101408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654f2d8b-8a06-488c-b78b-23548abba132 service placement] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:01.126837 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1025/2044] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:12:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:12:02.581383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8cefd5-07b6-41a1-9126-04211344f58b None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:02.583777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-6f8cefd5-07b6-41a1-9126-04211344f58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:02.681169 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8cefd5-07b6-41a1-9126-04211344f58b None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:02.686235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad1c7f-3870-491f-96fc-f59d0d0e24ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:02.687291 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad1c7f-3870-491f-96fc-f59d0d0e24ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:02.687815 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad1c7f-3870-491f-96fc-f59d0d0e24ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:02.688299 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad1c7f-3870-491f-96fc-f59d0d0e24ea admin admin] PATH_INFO: `/v3/users/208c0e97f7fe44ce8712580db0677bab` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:02.689719 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad1c7f-3870-491f-96fc-f59d0d0e24ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=208c0e97f7fe44ce8712580db0677bab)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:02.692025 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad1c7f-3870-491f-96fc-f59d0d0e24ea admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:02.768530 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-d4ad1c7f-3870-491f-96fc-f59d0d0e24ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '208c0e97f7fe44ce8712580db0677bab'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:02.778164 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-d4ad1c7f-3870-491f-96fc-f59d0d0e24ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '208c0e97f7fe44ce8712580db0677bab'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:02.779322 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-d4ad1c7f-3870-491f-96fc-f59d0d0e24ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '208c0e97f7fe44ce8712580db0677bab'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:02.780177 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-d4ad1c7f-3870-491f-96fc-f59d0d0e24ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '208c0e97f7fe44ce8712580db0677bab'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:02.793961 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-d4ad1c7f-3870-491f-96fc-f59d0d0e24ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '208c0e97f7fe44ce8712580db0677bab'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:02.848795 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1020/2045] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:12:02 2022] DELETE /identity/v3/users/208c0e97f7fe44ce8712580db0677bab => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:12:03.361740 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654f2d8b-8a06-488c-b78b-23548abba132 service placement] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:03.389327 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654f2d8b-8a06-488c-b78b-23548abba132 service placement] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:03.483811 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:12:03.500670 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba3052f-66d3-4642-9d65-e0f3baa10a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:03.501381 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba3052f-66d3-4642-9d65-e0f3baa10a5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:03.501761 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba3052f-66d3-4642-9d65-e0f3baa10a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:03.502132 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba3052f-66d3-4642-9d65-e0f3baa10a5d admin admin] PATH_INFO: `/v3/projects/ec8572a8f5484f5ca69cc125fc6025e9` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:03.503444 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba3052f-66d3-4642-9d65-e0f3baa10a5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec8572a8f5484f5ca69cc125fc6025e9)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:03.506022 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba3052f-66d3-4642-9d65-e0f3baa10a5d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:03.586759 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4ba3052f-66d3-4642-9d65-e0f3baa10a5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec8572a8f5484f5ca69cc125fc6025e9', 'request_id': 'req-4ba3052f-66d3-4642-9d65-e0f3baa10a5d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:03.594292 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4ba3052f-66d3-4642-9d65-e0f3baa10a5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec8572a8f5484f5ca69cc125fc6025e9', 'request_id': 'req-4ba3052f-66d3-4642-9d65-e0f3baa10a5d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:03.602948 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4ba3052f-66d3-4642-9d65-e0f3baa10a5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec8572a8f5484f5ca69cc125fc6025e9', 'request_id': 'req-4ba3052f-66d3-4642-9d65-e0f3baa10a5d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:03.622697 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4ba3052f-66d3-4642-9d65-e0f3baa10a5d admin admin] The token cache is being invalidate because project ec8572a8f5484f5ca69cc125fc6025e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:12:03.623351 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4ba3052f-66d3-4642-9d65-e0f3baa10a5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:03.627220 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4ba3052f-66d3-4642-9d65-e0f3baa10a5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:03.631641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1026/2046] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:12:03 2022] DELETE /identity/v3/projects/ec8572a8f5484f5ca69cc125fc6025e9 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:12:03.635700 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad1c7f-3870-491f-96fc-f59d0d0e24ea admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:03.637822 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-d4ad1c7f-3870-491f-96fc-f59d0d0e24ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:03.657003 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad1c7f-3870-491f-96fc-f59d0d0e24ea admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:03.758806 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4741a-649d-4399-9892-340731a438fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:03.760139 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4741a-649d-4399-9892-340731a438fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:03.760523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4741a-649d-4399-9892-340731a438fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:03.760897 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4741a-649d-4399-9892-340731a438fe admin admin] PATH_INFO: `/v3/users/f8b6d604fc594479a076d38a669ac600` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:03.762749 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4741a-649d-4399-9892-340731a438fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8b6d604fc594479a076d38a669ac600)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:03.764976 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4741a-649d-4399-9892-340731a438fe admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:03.781710 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:12:03.882138 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-51c4741a-649d-4399-9892-340731a438fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8b6d604fc594479a076d38a669ac600'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:03.892761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-51c4741a-649d-4399-9892-340731a438fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8b6d604fc594479a076d38a669ac600'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:03.893629 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-51c4741a-649d-4399-9892-340731a438fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8b6d604fc594479a076d38a669ac600'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:03.895370 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-51c4741a-649d-4399-9892-340731a438fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8b6d604fc594479a076d38a669ac600'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:03.911971 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-51c4741a-649d-4399-9892-340731a438fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8b6d604fc594479a076d38a669ac600'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:03.938661 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1021/2047] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:12:03 2022] DELETE /identity/v3/users/f8b6d604fc594479a076d38a669ac600 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:12:04.490283 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba3052f-66d3-4642-9d65-e0f3baa10a5d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:04.492059 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4ba3052f-66d3-4642-9d65-e0f3baa10a5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:04.544814 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba3052f-66d3-4642-9d65-e0f3baa10a5d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:04.636508 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084f5f2-d331-4820-b0fb-4ba96fbb4f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:04.637212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084f5f2-d331-4820-b0fb-4ba96fbb4f86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:04.637545 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084f5f2-d331-4820-b0fb-4ba96fbb4f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:04.637833 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084f5f2-d331-4820-b0fb-4ba96fbb4f86 admin admin] PATH_INFO: `/v3/projects/aada7626dfa44f5ebb843842c5859b51` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:04.642008 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084f5f2-d331-4820-b0fb-4ba96fbb4f86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aada7626dfa44f5ebb843842c5859b51)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:04.644274 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084f5f2-d331-4820-b0fb-4ba96fbb4f86 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:04.727941 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4084f5f2-d331-4820-b0fb-4ba96fbb4f86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aada7626dfa44f5ebb843842c5859b51', 'request_id': 'req-4084f5f2-d331-4820-b0fb-4ba96fbb4f86'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:04.743345 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4084f5f2-d331-4820-b0fb-4ba96fbb4f86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aada7626dfa44f5ebb843842c5859b51', 'request_id': 'req-4084f5f2-d331-4820-b0fb-4ba96fbb4f86'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:04.754795 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4084f5f2-d331-4820-b0fb-4ba96fbb4f86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aada7626dfa44f5ebb843842c5859b51', 'request_id': 'req-4084f5f2-d331-4820-b0fb-4ba96fbb4f86'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:04.778764 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4084f5f2-d331-4820-b0fb-4ba96fbb4f86 admin admin] The token cache is being invalidate because project aada7626dfa44f5ebb843842c5859b51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:12:04.779353 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4084f5f2-d331-4820-b0fb-4ba96fbb4f86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:04.781035 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-4084f5f2-d331-4820-b0fb-4ba96fbb4f86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:04.791521 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1027/2048] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:12:04 2022] DELETE /identity/v3/projects/aada7626dfa44f5ebb843842c5859b51 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:12:04.791731 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed38084-fc3e-4b28-bd57-dc14bf70d5f3 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:04.792243 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed38084-fc3e-4b28-bd57-dc14bf70d5f3 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:04.792355 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed38084-fc3e-4b28-bd57-dc14bf70d5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:04.837039 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-3ed38084-fc3e-4b28-bd57-dc14bf70d5f3 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:12:04.846152 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-3ed38084-fc3e-4b28-bd57-dc14bf70d5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:05.050016 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1022/2049] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:12:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:12:05.058449 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294aa73-0de1-40fb-85cd-66140a867340 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:05.058741 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294aa73-0de1-40fb-85cd-66140a867340 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:05.059022 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294aa73-0de1-40fb-85cd-66140a867340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:05.092918 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-7294aa73-0de1-40fb-85cd-66140a867340 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:12:05.094198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7294aa73-0de1-40fb-85cd-66140a867340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:05.115483 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1028/2050] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:12:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:12:05.119353 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed38084-fc3e-4b28-bd57-dc14bf70d5f3 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:05.129966 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed38084-fc3e-4b28-bd57-dc14bf70d5f3 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:05.138090 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce5f7d-39a8-4b7e-a829-5edd6309eca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:05.138878 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce5f7d-39a8-4b7e-a829-5edd6309eca1 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:05.139448 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce5f7d-39a8-4b7e-a829-5edd6309eca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:05.140059 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce5f7d-39a8-4b7e-a829-5edd6309eca1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:05.141464 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce5f7d-39a8-4b7e-a829-5edd6309eca1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:12:05.141879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce5f7d-39a8-4b7e-a829-5edd6309eca1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:05.145453 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce5f7d-39a8-4b7e-a829-5edd6309eca1 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:05.162232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1023/2051] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:12:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 13:12:05.165912 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294aa73-0de1-40fb-85cd-66140a867340 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:05.176627 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294aa73-0de1-40fb-85cd-66140a867340 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:05.182993 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4fa376-e42e-4e83-925d-ffa6ada88b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:05.183350 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4fa376-e42e-4e83-925d-ffa6ada88b7f admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:05.183738 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4fa376-e42e-4e83-925d-ffa6ada88b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:05.184133 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4fa376-e42e-4e83-925d-ffa6ada88b7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:05.184990 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4fa376-e42e-4e83-925d-ffa6ada88b7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:05.187568 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4fa376-e42e-4e83-925d-ffa6ada88b7f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:05.193625 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:12:05.248525 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1029/2052] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:12:05 2022] POST /identity/v3/projects => generated 377 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:12:05.252963 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce5f7d-39a8-4b7e-a829-5edd6309eca1 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:05.271132 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce5f7d-39a8-4b7e-a829-5edd6309eca1 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:05.457091 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487aaa19-d415-48b9-880c-20aea49348cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:05.458503 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487aaa19-d415-48b9-880c-20aea49348cd admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:05.458932 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487aaa19-d415-48b9-880c-20aea49348cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:05.459333 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487aaa19-d415-48b9-880c-20aea49348cd admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:05.460140 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487aaa19-d415-48b9-880c-20aea49348cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:05.462251 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487aaa19-d415-48b9-880c-20aea49348cd admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:05.522362 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1024/2053] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:12:05 2022] POST /identity/v3/users => generated 289 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:12:05.527524 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4fa376-e42e-4e83-925d-ffa6ada88b7f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:05.549872 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4fa376-e42e-4e83-925d-ffa6ada88b7f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:05.568091 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7bddf3-be9d-4787-891d-4ae1970808d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:05.569448 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7bddf3-be9d-4787-891d-4ae1970808d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:05.569973 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7bddf3-be9d-4787-891d-4ae1970808d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:05.570436 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7bddf3-be9d-4787-891d-4ae1970808d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:05.571332 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7bddf3-be9d-4787-891d-4ae1970808d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:05.573249 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7bddf3-be9d-4787-891d-4ae1970808d3 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:05.587760 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1030/2054] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:12:05.591496 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487aaa19-d415-48b9-880c-20aea49348cd admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:05.596611 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:12:05.604992 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487aaa19-d415-48b9-880c-20aea49348cd admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:05.614234 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90526342-92c7-4f61-8b73-52d8be17bb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:05.615015 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90526342-92c7-4f61-8b73-52d8be17bb81 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:05.615473 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90526342-92c7-4f61-8b73-52d8be17bb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:05.615922 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90526342-92c7-4f61-8b73-52d8be17bb81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:05.616733 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90526342-92c7-4f61-8b73-52d8be17bb81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:05.618871 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90526342-92c7-4f61-8b73-52d8be17bb81 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:05.633182 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1025/2055] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:12:05.637246 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7bddf3-be9d-4787-891d-4ae1970808d3 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:05.658039 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7bddf3-be9d-4787-891d-4ae1970808d3 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:05.666136 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb4753-02dd-4f15-944e-bfe96fa204f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:05.667747 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb4753-02dd-4f15-944e-bfe96fa204f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:05.668247 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb4753-02dd-4f15-944e-bfe96fa204f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:05.668715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb4753-02dd-4f15-944e-bfe96fa204f6 admin admin] PATH_INFO: `/v3/projects/0c535852cb7543d7b357a6333240d77c/users/d3d115e3e9b44f4ea188912a29f194ce/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:05.706433 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb4753-02dd-4f15-944e-bfe96fa204f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c535852cb7543d7b357a6333240d77c, user_id=d3d115e3e9b44f4ea188912a29f194ce, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:05.710316 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb4753-02dd-4f15-944e-bfe96fa204f6 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:05.730502 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1031/2056] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:12:05 2022] PUT /identity/v3/projects/0c535852cb7543d7b357a6333240d77c/users/d3d115e3e9b44f4ea188912a29f194ce/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:12:05.741044 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5023378-85c4-43ac-a6dc-8a1cfff8e93e None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:05.741421 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5023378-85c4-43ac-a6dc-8a1cfff8e93e None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:05.741719 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5023378-85c4-43ac-a6dc-8a1cfff8e93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:05.777216 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-c5023378-85c4-43ac-a6dc-8a1cfff8e93e None None] MFA Rules not processed for user `d3d115e3e9b44f4ea188912a29f194ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:12:05.778539 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-c5023378-85c4-43ac-a6dc-8a1cfff8e93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:05.882446 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1026/2057] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:12:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:12:05.886931 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb4753-02dd-4f15-944e-bfe96fa204f6 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:05.988528 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb4753-02dd-4f15-944e-bfe96fa204f6 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:05.993771 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c606a5-c115-4b4b-a9a9-9b7a7953eb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:05.994399 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c606a5-c115-4b4b-a9a9-9b7a7953eb60 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:05.994679 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c606a5-c115-4b4b-a9a9-9b7a7953eb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:05.994956 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c606a5-c115-4b4b-a9a9-9b7a7953eb60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:05.995717 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c606a5-c115-4b4b-a9a9-9b7a7953eb60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:05.997980 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c606a5-c115-4b4b-a9a9-9b7a7953eb60 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:06.042793 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1032/2058] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:12:05 2022] POST /identity/v3/projects => generated 377 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:12:06.047266 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5023378-85c4-43ac-a6dc-8a1cfff8e93e None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:06.065523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5023378-85c4-43ac-a6dc-8a1cfff8e93e None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:06.153386 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed16f1ab-eff9-4ea1-9746-dcb324982530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:06.154166 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16f1ab-eff9-4ea1-9746-dcb324982530 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:06.154447 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16f1ab-eff9-4ea1-9746-dcb324982530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:06.154730 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16f1ab-eff9-4ea1-9746-dcb324982530 admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:06.155414 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed16f1ab-eff9-4ea1-9746-dcb324982530 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:06.157771 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed16f1ab-eff9-4ea1-9746-dcb324982530 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:06.194524 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1027/2059] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:12:06 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:12:06.195832 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c606a5-c115-4b4b-a9a9-9b7a7953eb60 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:06.213543 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c606a5-c115-4b4b-a9a9-9b7a7953eb60 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:06.220189 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e8485d-f369-4aac-a900-2072867a0ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:06.221066 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8485d-f369-4aac-a900-2072867a0ee7 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:06.221555 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8485d-f369-4aac-a900-2072867a0ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:06.222038 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8485d-f369-4aac-a900-2072867a0ee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:06.222696 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e8485d-f369-4aac-a900-2072867a0ee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:06.224400 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e8485d-f369-4aac-a900-2072867a0ee7 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:06.241730 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1033/2060] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:12:06.245750 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed16f1ab-eff9-4ea1-9746-dcb324982530 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:06.258085 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed16f1ab-eff9-4ea1-9746-dcb324982530 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:06.263903 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca93ed9-f15a-448e-8977-73ec1f5dccfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:06.267517 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca93ed9-f15a-448e-8977-73ec1f5dccfd admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:06.267835 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca93ed9-f15a-448e-8977-73ec1f5dccfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:06.268099 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca93ed9-f15a-448e-8977-73ec1f5dccfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:06.268787 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca93ed9-f15a-448e-8977-73ec1f5dccfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:06.270975 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca93ed9-f15a-448e-8977-73ec1f5dccfd admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:06.286365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1028/2061] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:12:06.287522 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e8485d-f369-4aac-a900-2072867a0ee7 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:06.301742 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e8485d-f369-4aac-a900-2072867a0ee7 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:06.307551 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed271c-d2f4-48a2-900a-dccf96e608bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:06.311015 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed271c-d2f4-48a2-900a-dccf96e608bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:06.311242 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed271c-d2f4-48a2-900a-dccf96e608bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:06.311521 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed271c-d2f4-48a2-900a-dccf96e608bf admin admin] PATH_INFO: `/v3/projects/22f558a77aff450a83b7d744947e7a12/users/fc2907fcc4a44b03bcf9883917c944d5/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:06.336405 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed271c-d2f4-48a2-900a-dccf96e608bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=22f558a77aff450a83b7d744947e7a12, user_id=fc2907fcc4a44b03bcf9883917c944d5, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:06.339514 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed271c-d2f4-48a2-900a-dccf96e608bf admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:06.367141 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1034/2062] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:12:06 2022] PUT /identity/v3/projects/22f558a77aff450a83b7d744947e7a12/users/fc2907fcc4a44b03bcf9883917c944d5/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:12:06.369510 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd32e4c-cee5-4c89-8bf3-2ce65a2552c2 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:06.369990 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd32e4c-cee5-4c89-8bf3-2ce65a2552c2 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:06.370444 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd32e4c-cee5-4c89-8bf3-2ce65a2552c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:06.418547 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-9dd32e4c-cee5-4c89-8bf3-2ce65a2552c2 None None] MFA Rules not processed for user `fc2907fcc4a44b03bcf9883917c944d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:12:06.419860 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-9dd32e4c-cee5-4c89-8bf3-2ce65a2552c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:06.516732 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:12:06.553244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1029/2063] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:12:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:12:06.554115 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed271c-d2f4-48a2-900a-dccf96e608bf admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:06.677384 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed271c-d2f4-48a2-900a-dccf96e608bf admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:06.687756 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142cf553-5209-428d-b8a6-edfaba82090e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:06.688493 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142cf553-5209-428d-b8a6-edfaba82090e admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:06.688846 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142cf553-5209-428d-b8a6-edfaba82090e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:06.689325 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142cf553-5209-428d-b8a6-edfaba82090e admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:06.690041 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142cf553-5209-428d-b8a6-edfaba82090e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:06.692276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142cf553-5209-428d-b8a6-edfaba82090e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:06.700188 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:12:06.756304 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1035/2064] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:12:06 2022] POST /identity/v3/projects => generated 377 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:12:06.760184 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd32e4c-cee5-4c89-8bf3-2ce65a2552c2 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:06.774288 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd32e4c-cee5-4c89-8bf3-2ce65a2552c2 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:06.940947 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d60dd-c0ef-4793-8173-ffa116180339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:06.947088 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d60dd-c0ef-4793-8173-ffa116180339 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:06.947088 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d60dd-c0ef-4793-8173-ffa116180339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:06.947088 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d60dd-c0ef-4793-8173-ffa116180339 admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:06.947088 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d60dd-c0ef-4793-8173-ffa116180339 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:06.952564 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d60dd-c0ef-4793-8173-ffa116180339 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:07.008099 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1030/2065] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:12:06 2022] POST /identity/v3/users => generated 295 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:12:07.015686 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142cf553-5209-428d-b8a6-edfaba82090e admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:07.033937 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142cf553-5209-428d-b8a6-edfaba82090e admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:07.043571 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e2e1d-1ec9-4ad6-9cbe-638c47571a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:07.044304 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e2e1d-1ec9-4ad6-9cbe-638c47571a0b admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:07.044584 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e2e1d-1ec9-4ad6-9cbe-638c47571a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:07.044859 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e2e1d-1ec9-4ad6-9cbe-638c47571a0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:07.046016 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3e2e1d-1ec9-4ad6-9cbe-638c47571a0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:07.048125 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3e2e1d-1ec9-4ad6-9cbe-638c47571a0b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:07.067380 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1036/2066] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:12:07.068203 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d60dd-c0ef-4793-8173-ffa116180339 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:07.084581 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d60dd-c0ef-4793-8173-ffa116180339 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:07.096577 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef39fce-2d25-47aa-a768-96757ee1c845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:07.097371 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef39fce-2d25-47aa-a768-96757ee1c845 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:07.099814 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef39fce-2d25-47aa-a768-96757ee1c845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:07.100209 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef39fce-2d25-47aa-a768-96757ee1c845 admin admin] PATH_INFO: `/v3/projects/5fdd9aae53b64c118f650a802e190340/users/18559e952d3f4b72bb34cfd3241c8c4b/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:07.139115 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef39fce-2d25-47aa-a768-96757ee1c845 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fdd9aae53b64c118f650a802e190340, user_id=18559e952d3f4b72bb34cfd3241c8c4b, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:07.141905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef39fce-2d25-47aa-a768-96757ee1c845 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:07.169066 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1031/2067] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:12:07 2022] PUT /identity/v3/projects/5fdd9aae53b64c118f650a802e190340/users/18559e952d3f4b72bb34cfd3241c8c4b/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:12:07.172844 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e2e1d-1ec9-4ad6-9cbe-638c47571a0b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:07.192731 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e2e1d-1ec9-4ad6-9cbe-638c47571a0b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:07.335700 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081ba21-4b32-47cb-ae74-24934a1ca281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:07.336312 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081ba21-4b32-47cb-ae74-24934a1ca281 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:07.336593 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081ba21-4b32-47cb-ae74-24934a1ca281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:07.336865 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081ba21-4b32-47cb-ae74-24934a1ca281 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:07.337601 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081ba21-4b32-47cb-ae74-24934a1ca281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:07.339353 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081ba21-4b32-47cb-ae74-24934a1ca281 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:07.354295 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1037/2068] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:12:07.355555 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef39fce-2d25-47aa-a768-96757ee1c845 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:07.378284 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef39fce-2d25-47aa-a768-96757ee1c845 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:07.401859 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f46df7f-b737-45b4-b796-dfa6705c4343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:07.402948 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46df7f-b737-45b4-b796-dfa6705c4343 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:07.403388 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46df7f-b737-45b4-b796-dfa6705c4343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:07.403767 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46df7f-b737-45b4-b796-dfa6705c4343 admin admin] PATH_INFO: `/v3/projects/5fdd9aae53b64c118f650a802e190340/users/18559e952d3f4b72bb34cfd3241c8c4b/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:07.406014 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f46df7f-b737-45b4-b796-dfa6705c4343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fdd9aae53b64c118f650a802e190340, user_id=18559e952d3f4b72bb34cfd3241c8c4b, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:07.408952 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f46df7f-b737-45b4-b796-dfa6705c4343 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:07.425176 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1032/2069] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:12:07 2022] PUT /identity/v3/projects/5fdd9aae53b64c118f650a802e190340/users/18559e952d3f4b72bb34cfd3241c8c4b/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:12:07.436726 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77416685-dc5b-4e61-9f6b-a83403cf9b50 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:07.437028 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77416685-dc5b-4e61-9f6b-a83403cf9b50 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:07.437417 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77416685-dc5b-4e61-9f6b-a83403cf9b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:07.469556 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-77416685-dc5b-4e61-9f6b-a83403cf9b50 None None] MFA Rules not processed for user `18559e952d3f4b72bb34cfd3241c8c4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:12:07.470846 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-77416685-dc5b-4e61-9f6b-a83403cf9b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:07.551867 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1038/2070] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:12:07 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:12:07.565843 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f46df7f-b737-45b4-b796-dfa6705c4343 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:07.568152 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-7f46df7f-b737-45b4-b796-dfa6705c4343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:07.633161 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f46df7f-b737-45b4-b796-dfa6705c4343 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:07.637742 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb0b88-2fcf-4878-bb92-91cfe8a77f80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:07.638494 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb0b88-2fcf-4878-bb92-91cfe8a77f80 service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:07.638844 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb0b88-2fcf-4878-bb92-91cfe8a77f80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:07.639151 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb0b88-2fcf-4878-bb92-91cfe8a77f80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:07.708416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:12:07.710888 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcb0b88-2fcf-4878-bb92-91cfe8a77f80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:07.712437 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcb0b88-2fcf-4878-bb92-91cfe8a77f80 service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:07.725744 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1033/2071] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:12:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:12:09.669904 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77416685-dc5b-4e61-9f6b-a83403cf9b50 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:09.671888 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-77416685-dc5b-4e61-9f6b-a83403cf9b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:09.692745 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77416685-dc5b-4e61-9f6b-a83403cf9b50 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:09.696586 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0463c83-3899-4fb9-86c6-2830488aa383 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:09.697842 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0463c83-3899-4fb9-86c6-2830488aa383 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:09.698127 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0463c83-3899-4fb9-86c6-2830488aa383 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:09.698406 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0463c83-3899-4fb9-86c6-2830488aa383 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:09.778841 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:12:09.786797 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0463c83-3899-4fb9-86c6-2830488aa383 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:09.788487 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0463c83-3899-4fb9-86c6-2830488aa383 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:09.800491 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1039/2072] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:12:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:12:10.651206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb0b88-2fcf-4878-bb92-91cfe8a77f80 service cinder] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:10.661215 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb0b88-2fcf-4878-bb92-91cfe8a77f80 service cinder] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:10.665815 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a834c3-7b09-4261-9721-fc8a9ce3eb55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:10.666802 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a834c3-7b09-4261-9721-fc8a9ce3eb55 service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:10.667511 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a834c3-7b09-4261-9721-fc8a9ce3eb55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:10.667635 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a834c3-7b09-4261-9721-fc8a9ce3eb55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:10.678764 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a834c3-7b09-4261-9721-fc8a9ce3eb55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:10.680994 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a834c3-7b09-4261-9721-fc8a9ce3eb55 service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:10.701106 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1034/2073] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:12:10 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:12:11.595853 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8005ac-7873-4777-925c-2df0a719b1c4 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:11.596263 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8005ac-7873-4777-925c-2df0a719b1c4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:11.596263 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8005ac-7873-4777-925c-2df0a719b1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:11.626945 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-ff8005ac-7873-4777-925c-2df0a719b1c4 None None] MFA Rules not processed for user `b6ae674f0efc445fa970fca64e13893a`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:12:11.628438 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ff8005ac-7873-4777-925c-2df0a719b1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:11.706254 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1040/2074] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Oct 13 13:12:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:12:11.720828 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a834c3-7b09-4261-9721-fc8a9ce3eb55 service cinder] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:11.722352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-79a834c3-7b09-4261-9721-fc8a9ce3eb55 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:11.813638 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a834c3-7b09-4261-9721-fc8a9ce3eb55 service cinder] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:11.819941 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbfcb1c-5f79-4933-960f-468407fc9cfc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526000341fd146db8c661f1fd14d4801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:11.820861 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbfcb1c-5f79-4933-960f-468407fc9cfc service swift] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:11.821233 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbfcb1c-5f79-4933-960f-468407fc9cfc service swift] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:11.821595 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbfcb1c-5f79-4933-960f-468407fc9cfc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:11.830565 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbfcb1c-5f79-4933-960f-468407fc9cfc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:11.833057 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbfcb1c-5f79-4933-960f-468407fc9cfc service swift] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:11.843438 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1035/2075] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 13 13:12:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 13 13:12:13.527578 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef41a7a-9803-47aa-b2b8-816ead58d502 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:13.528155 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef41a7a-9803-47aa-b2b8-816ead58d502 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:13.528155 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef41a7a-9803-47aa-b2b8-816ead58d502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:13.561987 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-cef41a7a-9803-47aa-b2b8-816ead58d502 None None] MFA Rules not processed for user `454203f47b434cca952a1cef56de559f`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:12:13.563243 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-cef41a7a-9803-47aa-b2b8-816ead58d502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:13.762072 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1041/2076] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:12:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:12:15.331456 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e5f8f-e56e-4dba-8051-42bc37faf9f4 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:15.332046 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e5f8f-e56e-4dba-8051-42bc37faf9f4 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:15.332806 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e5f8f-e56e-4dba-8051-42bc37faf9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:15.376494 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-a37e5f8f-e56e-4dba-8051-42bc37faf9f4 None None] MFA Rules not processed for user `7d89667e52b24295ba619203c02c7bd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:12:15.379864 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-a37e5f8f-e56e-4dba-8051-42bc37faf9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:15.514687 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1036/2077] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:12:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:12:16.574822 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef41a7a-9803-47aa-b2b8-816ead58d502 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:16.680216 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef41a7a-9803-47aa-b2b8-816ead58d502 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:16.685894 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5da254-2db0-489f-8fb5-1f10e71d3dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:16.687620 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5da254-2db0-489f-8fb5-1f10e71d3dbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:16.688142 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5da254-2db0-489f-8fb5-1f10e71d3dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:16.688616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5da254-2db0-489f-8fb5-1f10e71d3dbb admin admin] PATH_INFO: `/v3/users/d3d115e3e9b44f4ea188912a29f194ce` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:16.690334 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5da254-2db0-489f-8fb5-1f10e71d3dbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3d115e3e9b44f4ea188912a29f194ce)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:16.694020 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5da254-2db0-489f-8fb5-1f10e71d3dbb admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:16.831847 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:12:16.836427 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ce5da254-2db0-489f-8fb5-1f10e71d3dbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3d115e3e9b44f4ea188912a29f194ce'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:16.860692 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ce5da254-2db0-489f-8fb5-1f10e71d3dbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3d115e3e9b44f4ea188912a29f194ce'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:16.861858 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ce5da254-2db0-489f-8fb5-1f10e71d3dbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3d115e3e9b44f4ea188912a29f194ce'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:16.881076 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ce5da254-2db0-489f-8fb5-1f10e71d3dbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3d115e3e9b44f4ea188912a29f194ce'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:16.944721 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ce5da254-2db0-489f-8fb5-1f10e71d3dbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3d115e3e9b44f4ea188912a29f194ce'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:16.971378 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1042/2078] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:12:16 2022] DELETE /identity/v3/users/d3d115e3e9b44f4ea188912a29f194ce => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:12:16.976849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45033cb-e6bd-4df7-ac3e-ae34cebdc687 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:16.977317 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45033cb-e6bd-4df7-ac3e-ae34cebdc687 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:16.977940 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45033cb-e6bd-4df7-ac3e-ae34cebdc687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:17.035941 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-a45033cb-e6bd-4df7-ac3e-ae34cebdc687 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:12:17.041259 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-a45033cb-e6bd-4df7-ac3e-ae34cebdc687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:17.219394 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1037/2079] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:12:17.231720 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5da254-2db0-489f-8fb5-1f10e71d3dbb admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:17.233641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ce5da254-2db0-489f-8fb5-1f10e71d3dbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:17.390232 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5da254-2db0-489f-8fb5-1f10e71d3dbb admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:17.411331 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5029b-2b2d-4670-9880-c3a9de82dd18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:17.412195 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5029b-2b2d-4670-9880-c3a9de82dd18 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:17.412712 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5029b-2b2d-4670-9880-c3a9de82dd18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:17.413729 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5029b-2b2d-4670-9880-c3a9de82dd18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:17.427458 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5029b-2b2d-4670-9880-c3a9de82dd18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:17.429152 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5029b-2b2d-4670-9880-c3a9de82dd18 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:17.457542 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1043/2080] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:12:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:12:17.820762 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45033cb-e6bd-4df7-ac3e-ae34cebdc687 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:17.822436 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-a45033cb-e6bd-4df7-ac3e-ae34cebdc687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:17.839758 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45033cb-e6bd-4df7-ac3e-ae34cebdc687 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:17.850365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff99c8f2-b34a-4a59-b0c5-18849d940136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:17.850703 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99c8f2-b34a-4a59-b0c5-18849d940136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:17.850986 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99c8f2-b34a-4a59-b0c5-18849d940136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:17.851317 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff99c8f2-b34a-4a59-b0c5-18849d940136 admin admin] PATH_INFO: `/v3/projects/0c535852cb7543d7b357a6333240d77c` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:17.852461 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff99c8f2-b34a-4a59-b0c5-18849d940136 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c535852cb7543d7b357a6333240d77c)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:17.879678 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff99c8f2-b34a-4a59-b0c5-18849d940136 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:17.898900 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:12:17.955069 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-ff99c8f2-b34a-4a59-b0c5-18849d940136 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c535852cb7543d7b357a6333240d77c', 'request_id': 'req-ff99c8f2-b34a-4a59-b0c5-18849d940136'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:17.964259 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-ff99c8f2-b34a-4a59-b0c5-18849d940136 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c535852cb7543d7b357a6333240d77c', 'request_id': 'req-ff99c8f2-b34a-4a59-b0c5-18849d940136'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:17.979403 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-ff99c8f2-b34a-4a59-b0c5-18849d940136 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c535852cb7543d7b357a6333240d77c', 'request_id': 'req-ff99c8f2-b34a-4a59-b0c5-18849d940136'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:18.010876 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-ff99c8f2-b34a-4a59-b0c5-18849d940136 admin admin] The token cache is being invalidate because project 0c535852cb7543d7b357a6333240d77c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:12:18.011212 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-ff99c8f2-b34a-4a59-b0c5-18849d940136 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:18.012226 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-ff99c8f2-b34a-4a59-b0c5-18849d940136 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:18.013730 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1038/2081] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:12:17 2022] DELETE /identity/v3/projects/0c535852cb7543d7b357a6333240d77c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:12:18.018017 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5029b-2b2d-4670-9880-c3a9de82dd18 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:18.034189 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5029b-2b2d-4670-9880-c3a9de82dd18 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:18.136246 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be206b-e20f-423f-8270-5e615c0d684b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:18.137252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be206b-e20f-423f-8270-5e615c0d684b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:18.137636 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be206b-e20f-423f-8270-5e615c0d684b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:18.138014 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be206b-e20f-423f-8270-5e615c0d684b admin admin] PATH_INFO: `/v3/users/fc2907fcc4a44b03bcf9883917c944d5` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:18.144022 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15be206b-e20f-423f-8270-5e615c0d684b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc2907fcc4a44b03bcf9883917c944d5)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:18.156654 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15be206b-e20f-423f-8270-5e615c0d684b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:18.228624 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-15be206b-e20f-423f-8270-5e615c0d684b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc2907fcc4a44b03bcf9883917c944d5'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:18.240056 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-15be206b-e20f-423f-8270-5e615c0d684b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc2907fcc4a44b03bcf9883917c944d5'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:18.241132 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-15be206b-e20f-423f-8270-5e615c0d684b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc2907fcc4a44b03bcf9883917c944d5'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:18.256267 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-15be206b-e20f-423f-8270-5e615c0d684b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc2907fcc4a44b03bcf9883917c944d5'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:18.271520 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:12:18.302979 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-15be206b-e20f-423f-8270-5e615c0d684b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc2907fcc4a44b03bcf9883917c944d5'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:18.317944 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1044/2082] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:12:18 2022] DELETE /identity/v3/users/fc2907fcc4a44b03bcf9883917c944d5 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:12:18.540857 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b17bd-8640-41fd-aa7b-294c380839d9 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:18.541144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b17bd-8640-41fd-aa7b-294c380839d9 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:18.541424 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b17bd-8640-41fd-aa7b-294c380839d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:18.604744 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-8b5b17bd-8640-41fd-aa7b-294c380839d9 None None] MFA Rules not processed for user `b6ae674f0efc445fa970fca64e13893a`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:12:18.605975 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-8b5b17bd-8640-41fd-aa7b-294c380839d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:18.746556 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1039/2083] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Oct 13 13:12:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:12:18.797948 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be206b-e20f-423f-8270-5e615c0d684b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:18.799634 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-15be206b-e20f-423f-8270-5e615c0d684b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:18.999693 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be206b-e20f-423f-8270-5e615c0d684b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:19.018098 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962dbb69-a8d7-42b6-b01b-66eccdba14c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526000341fd146db8c661f1fd14d4801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:19.018816 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962dbb69-a8d7-42b6-b01b-66eccdba14c8 service swift] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:19.019048 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962dbb69-a8d7-42b6-b01b-66eccdba14c8 service swift] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:19.019876 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962dbb69-a8d7-42b6-b01b-66eccdba14c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:19.032824 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962dbb69-a8d7-42b6-b01b-66eccdba14c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:19.034664 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962dbb69-a8d7-42b6-b01b-66eccdba14c8 service swift] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:19.046084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1045/2084] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 13 13:12:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 13 13:12:19.416594 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b17bd-8640-41fd-aa7b-294c380839d9 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:19.418872 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-8b5b17bd-8640-41fd-aa7b-294c380839d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:19.575775 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b17bd-8640-41fd-aa7b-294c380839d9 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:19.582354 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc73681-3cc2-4221-b919-062099b88c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:19.582989 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc73681-3cc2-4221-b919-062099b88c47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:19.583594 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc73681-3cc2-4221-b919-062099b88c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:19.583871 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc73681-3cc2-4221-b919-062099b88c47 admin admin] PATH_INFO: `/v3/projects/22f558a77aff450a83b7d744947e7a12` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:19.585644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc73681-3cc2-4221-b919-062099b88c47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22f558a77aff450a83b7d744947e7a12)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:19.588527 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc73681-3cc2-4221-b919-062099b88c47 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:19.668151 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:12:19.670827 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-7bc73681-3cc2-4221-b919-062099b88c47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22f558a77aff450a83b7d744947e7a12', 'request_id': 'req-7bc73681-3cc2-4221-b919-062099b88c47'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:19.684711 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-7bc73681-3cc2-4221-b919-062099b88c47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22f558a77aff450a83b7d744947e7a12', 'request_id': 'req-7bc73681-3cc2-4221-b919-062099b88c47'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:19.716306 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-7bc73681-3cc2-4221-b919-062099b88c47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22f558a77aff450a83b7d744947e7a12', 'request_id': 'req-7bc73681-3cc2-4221-b919-062099b88c47'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:19.758982 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-7bc73681-3cc2-4221-b919-062099b88c47 admin admin] The token cache is being invalidate because project 22f558a77aff450a83b7d744947e7a12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:12:19.761501 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-7bc73681-3cc2-4221-b919-062099b88c47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:19.767022 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-7bc73681-3cc2-4221-b919-062099b88c47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:19.774476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1040/2085] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:12:19 2022] DELETE /identity/v3/projects/22f558a77aff450a83b7d744947e7a12 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:12:19.776058 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962dbb69-a8d7-42b6-b01b-66eccdba14c8 service swift] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:19.800066 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962dbb69-a8d7-42b6-b01b-66eccdba14c8 service swift] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:19.966910 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966f3ab-35ed-46d0-bfb4-dc7e2b8e144f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:19.967587 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966f3ab-35ed-46d0-bfb4-dc7e2b8e144f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:19.967871 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966f3ab-35ed-46d0-bfb4-dc7e2b8e144f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:19.968149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966f3ab-35ed-46d0-bfb4-dc7e2b8e144f admin admin] PATH_INFO: `/v3/users/18559e952d3f4b72bb34cfd3241c8c4b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:19.974921 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5966f3ab-35ed-46d0-bfb4-dc7e2b8e144f admin admin] RBAC: Authorizing `identity:delete_user(user_id=18559e952d3f4b72bb34cfd3241c8c4b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:19.976936 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5966f3ab-35ed-46d0-bfb4-dc7e2b8e144f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:20.024537 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed2d2f-bdc6-4149-95c7-b0435524de6b None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:20.025049 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed2d2f-bdc6-4149-95c7-b0435524de6b None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:20.025392 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed2d2f-bdc6-4149-95c7-b0435524de6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:20.073640 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-2eed2d2f-bdc6-4149-95c7-b0435524de6b None None] MFA Rules not processed for user `b6ae674f0efc445fa970fca64e13893a`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:12:20.075229 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-2eed2d2f-bdc6-4149-95c7-b0435524de6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:20.082412 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5966f3ab-35ed-46d0-bfb4-dc7e2b8e144f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18559e952d3f4b72bb34cfd3241c8c4b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:20.091225 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5966f3ab-35ed-46d0-bfb4-dc7e2b8e144f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18559e952d3f4b72bb34cfd3241c8c4b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:20.091934 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5966f3ab-35ed-46d0-bfb4-dc7e2b8e144f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18559e952d3f4b72bb34cfd3241c8c4b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:20.103571 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5966f3ab-35ed-46d0-bfb4-dc7e2b8e144f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18559e952d3f4b72bb34cfd3241c8c4b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:20.131645 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:12:20.134761 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5966f3ab-35ed-46d0-bfb4-dc7e2b8e144f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18559e952d3f4b72bb34cfd3241c8c4b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:20.158908 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1046/2086] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:12:19 2022] DELETE /identity/v3/users/18559e952d3f4b72bb34cfd3241c8c4b => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:12:20.301434 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1041/2087] 10.222.0.23 () {58 vars in 967 bytes} [Thu Oct 13 13:12:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:12:20.331078 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966f3ab-35ed-46d0-bfb4-dc7e2b8e144f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:20.333294 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5966f3ab-35ed-46d0-bfb4-dc7e2b8e144f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:20.483491 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966f3ab-35ed-46d0-bfb4-dc7e2b8e144f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:20.490239 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6930fa9e-d03c-463e-b781-71673289f01d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526000341fd146db8c661f1fd14d4801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:20.491472 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930fa9e-d03c-463e-b781-71673289f01d service swift] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:20.491472 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930fa9e-d03c-463e-b781-71673289f01d service swift] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:20.491718 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930fa9e-d03c-463e-b781-71673289f01d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:20.590115 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6930fa9e-d03c-463e-b781-71673289f01d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:20.591844 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6930fa9e-d03c-463e-b781-71673289f01d service swift] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:20.604469 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1047/2088] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 13 13:12:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 13 13:12:20.640401 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed2d2f-bdc6-4149-95c7-b0435524de6b None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:20.642794 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-2eed2d2f-bdc6-4149-95c7-b0435524de6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:20.662822 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed2d2f-bdc6-4149-95c7-b0435524de6b None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:20.737636 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734895c-0297-4d8f-8600-fdf4ad24d2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:20.739450 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734895c-0297-4d8f-8600-fdf4ad24d2f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:20.739962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734895c-0297-4d8f-8600-fdf4ad24d2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:20.740471 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734895c-0297-4d8f-8600-fdf4ad24d2f9 admin admin] PATH_INFO: `/v3/projects/5fdd9aae53b64c118f650a802e190340` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:20.741756 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5734895c-0297-4d8f-8600-fdf4ad24d2f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fdd9aae53b64c118f650a802e190340)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:20.743975 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5734895c-0297-4d8f-8600-fdf4ad24d2f9 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:20.849563 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-5734895c-0297-4d8f-8600-fdf4ad24d2f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fdd9aae53b64c118f650a802e190340', 'request_id': 'req-5734895c-0297-4d8f-8600-fdf4ad24d2f9'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:20.866487 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-5734895c-0297-4d8f-8600-fdf4ad24d2f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fdd9aae53b64c118f650a802e190340', 'request_id': 'req-5734895c-0297-4d8f-8600-fdf4ad24d2f9'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:20.875587 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-5734895c-0297-4d8f-8600-fdf4ad24d2f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fdd9aae53b64c118f650a802e190340', 'request_id': 'req-5734895c-0297-4d8f-8600-fdf4ad24d2f9'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:20.893848 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-5734895c-0297-4d8f-8600-fdf4ad24d2f9 admin admin] The token cache is being invalidate because project 5fdd9aae53b64c118f650a802e190340 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:12:20.894162 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-5734895c-0297-4d8f-8600-fdf4ad24d2f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:20.895536 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-5734895c-0297-4d8f-8600-fdf4ad24d2f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:20.896892 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1042/2089] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:12:20 2022] DELETE /identity/v3/projects/5fdd9aae53b64c118f650a802e190340 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:12:21.004707 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be2c8f-695a-462f-8ee3-174d7c9ab429 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:21.005319 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be2c8f-695a-462f-8ee3-174d7c9ab429 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:21.005596 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be2c8f-695a-462f-8ee3-174d7c9ab429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:21.034273 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f0be2c8f-695a-462f-8ee3-174d7c9ab429 None None] MFA Rules not processed for user `b6ae674f0efc445fa970fca64e13893a`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:12:21.036526 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f0be2c8f-695a-462f-8ee3-174d7c9ab429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:21.119965 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1048/2090] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Oct 13 13:12:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:12:21.133863 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734895c-0297-4d8f-8600-fdf4ad24d2f9 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:21.135778 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-5734895c-0297-4d8f-8600-fdf4ad24d2f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:21.151906 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:12:21.224257 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734895c-0297-4d8f-8600-fdf4ad24d2f9 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:21.229403 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47ef22-8bbf-4a45-8cef-750c1751e74d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526000341fd146db8c661f1fd14d4801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:21.230007 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47ef22-8bbf-4a45-8cef-750c1751e74d service swift] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:21.230282 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47ef22-8bbf-4a45-8cef-750c1751e74d service swift] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:21.230582 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47ef22-8bbf-4a45-8cef-750c1751e74d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:21.241898 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47ef22-8bbf-4a45-8cef-750c1751e74d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:21.243707 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47ef22-8bbf-4a45-8cef-750c1751e74d service swift] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:21.253950 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1043/2091] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 13 13:12:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 13 13:12:21.984607 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c373218-5b87-4aa0-ad77-c3cbba203b1c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:21.984607 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c373218-5b87-4aa0-ad77-c3cbba203b1c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:21.984607 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c373218-5b87-4aa0-ad77-c3cbba203b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:22.008438 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-7c373218-5b87-4aa0-ad77-c3cbba203b1c None None] MFA Rules not processed for user `b6ae674f0efc445fa970fca64e13893a`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:12:22.010021 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7c373218-5b87-4aa0-ad77-c3cbba203b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:22.023961 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1049/2092] 10.222.0.23 () {58 vars in 967 bytes} [Thu Oct 13 13:12:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:12:22.037214 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47ef22-8bbf-4a45-8cef-750c1751e74d service swift] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:22.050055 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47ef22-8bbf-4a45-8cef-750c1751e74d service swift] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:22.058277 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb96540-82c5-467f-95ce-d44e44c443f8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526000341fd146db8c661f1fd14d4801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:22.059035 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb96540-82c5-467f-95ce-d44e44c443f8 service swift] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:22.059522 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb96540-82c5-467f-95ce-d44e44c443f8 service swift] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:22.059962 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb96540-82c5-467f-95ce-d44e44c443f8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:22.072503 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb96540-82c5-467f-95ce-d44e44c443f8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:22.074384 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb96540-82c5-467f-95ce-d44e44c443f8 service swift] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:22.087244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1044/2093] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 13 13:12:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 13 13:12:22.136432 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae83fd-7826-476a-81e8-fe2d2e80ce74 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:22.136979 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae83fd-7826-476a-81e8-fe2d2e80ce74 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:22.136979 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae83fd-7826-476a-81e8-fe2d2e80ce74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:22.168430 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-76ae83fd-7826-476a-81e8-fe2d2e80ce74 None None] MFA Rules not processed for user `8569afc405f24a0993a023c818a0c272`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:12:22.170429 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-76ae83fd-7826-476a-81e8-fe2d2e80ce74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:22.219551 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:12:22.285778 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1050/2094] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:12:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:12:22.610753 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc2b32-9f7b-42a6-8399-b14d3f2e6536 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:22.610926 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc2b32-9f7b-42a6-8399-b14d3f2e6536 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:22.611446 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc2b32-9f7b-42a6-8399-b14d3f2e6536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:22.646973 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-25cc2b32-9f7b-42a6-8399-b14d3f2e6536 None None] MFA Rules not processed for user `b6ae674f0efc445fa970fca64e13893a`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:12:22.646973 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-25cc2b32-9f7b-42a6-8399-b14d3f2e6536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:22.673338 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1045/2095] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Oct 13 13:12:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:12:22.687430 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae83fd-7826-476a-81e8-fe2d2e80ce74 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:22.704167 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae83fd-7826-476a-81e8-fe2d2e80ce74 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:22.713939 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc6f8a-055a-424c-87b8-0079a5be80c4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526000341fd146db8c661f1fd14d4801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:22.715041 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc6f8a-055a-424c-87b8-0079a5be80c4 service swift] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:22.715499 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc6f8a-055a-424c-87b8-0079a5be80c4 service swift] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:22.715959 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc6f8a-055a-424c-87b8-0079a5be80c4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:22.728152 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc6f8a-055a-424c-87b8-0079a5be80c4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:22.729986 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc6f8a-055a-424c-87b8-0079a5be80c4 service swift] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:22.740994 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1051/2096] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 13 13:12:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 13 13:12:25.349800 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc2b32-9f7b-42a6-8399-b14d3f2e6536 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:25.352656 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-25cc2b32-9f7b-42a6-8399-b14d3f2e6536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:25.569839 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc2b32-9f7b-42a6-8399-b14d3f2e6536 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:25.582587 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e45c0-8173-442c-a324-4a06659e3443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:25.584192 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e45c0-8173-442c-a324-4a06659e3443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:25.584776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e45c0-8173-442c-a324-4a06659e3443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:25.585206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e45c0-8173-442c-a324-4a06659e3443 admin admin] PATH_INFO: `/v3/users/fd5dc5471e5246f4878f1d91d59b68f2` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:25.590599 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242e45c0-8173-442c-a324-4a06659e3443 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd5dc5471e5246f4878f1d91d59b68f2)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:25.592886 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242e45c0-8173-442c-a324-4a06659e3443 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:25.728228 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-242e45c0-8173-442c-a324-4a06659e3443 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd5dc5471e5246f4878f1d91d59b68f2'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:25.759144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-242e45c0-8173-442c-a324-4a06659e3443 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd5dc5471e5246f4878f1d91d59b68f2'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:25.762503 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-242e45c0-8173-442c-a324-4a06659e3443 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd5dc5471e5246f4878f1d91d59b68f2'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:25.767175 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-242e45c0-8173-442c-a324-4a06659e3443 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd5dc5471e5246f4878f1d91d59b68f2'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:25.800267 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-242e45c0-8173-442c-a324-4a06659e3443 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd5dc5471e5246f4878f1d91d59b68f2'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:25.836243 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:12:25.850717 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1046/2097] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:12:25 2022] DELETE /identity/v3/users/fd5dc5471e5246f4878f1d91d59b68f2 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:12:25.852963 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b79e63-17fe-4888-9c57-35bdf6e905a0 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:25.853478 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b79e63-17fe-4888-9c57-35bdf6e905a0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:25.853580 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b79e63-17fe-4888-9c57-35bdf6e905a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:25.980497 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-63b79e63-17fe-4888-9c57-35bdf6e905a0 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:12:25.981916 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-63b79e63-17fe-4888-9c57-35bdf6e905a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:26.171119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1052/2098] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:12:26.212700 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e45c0-8173-442c-a324-4a06659e3443 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:26.214865 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-242e45c0-8173-442c-a324-4a06659e3443 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:26.386410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e45c0-8173-442c-a324-4a06659e3443 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:26.408125 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b17fe-2a39-455c-b14c-5e6a5f735565 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:26.410274 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b17fe-2a39-455c-b14c-5e6a5f735565 service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:26.410716 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b17fe-2a39-455c-b14c-5e6a5f735565 service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:26.413862 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b17fe-2a39-455c-b14c-5e6a5f735565 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:26.438375 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b17fe-2a39-455c-b14c-5e6a5f735565 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:26.442748 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b17fe-2a39-455c-b14c-5e6a5f735565 service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:26.479209 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1047/2099] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:12:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:12:27.836546 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b79e63-17fe-4888-9c57-35bdf6e905a0 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:27.838700 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-63b79e63-17fe-4888-9c57-35bdf6e905a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:27.852312 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b79e63-17fe-4888-9c57-35bdf6e905a0 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:27.858594 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edee48d-5b66-4ca9-9fad-fc28df2200f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:27.859392 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edee48d-5b66-4ca9-9fad-fc28df2200f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:27.859787 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edee48d-5b66-4ca9-9fad-fc28df2200f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:27.860161 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edee48d-5b66-4ca9-9fad-fc28df2200f1 admin admin] PATH_INFO: `/v3/projects/e74136856dcf42c48689e6976f16caad` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:27.861940 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edee48d-5b66-4ca9-9fad-fc28df2200f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e74136856dcf42c48689e6976f16caad)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:27.863848 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edee48d-5b66-4ca9-9fad-fc28df2200f1 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:27.933778 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0edee48d-5b66-4ca9-9fad-fc28df2200f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e74136856dcf42c48689e6976f16caad', 'request_id': 'req-0edee48d-5b66-4ca9-9fad-fc28df2200f1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:27.939756 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0edee48d-5b66-4ca9-9fad-fc28df2200f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e74136856dcf42c48689e6976f16caad', 'request_id': 'req-0edee48d-5b66-4ca9-9fad-fc28df2200f1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:27.948682 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0edee48d-5b66-4ca9-9fad-fc28df2200f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e74136856dcf42c48689e6976f16caad', 'request_id': 'req-0edee48d-5b66-4ca9-9fad-fc28df2200f1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:27.962099 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0edee48d-5b66-4ca9-9fad-fc28df2200f1 admin admin] The token cache is being invalidate because project e74136856dcf42c48689e6976f16caad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:12:27.962472 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0edee48d-5b66-4ca9-9fad-fc28df2200f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:27.963075 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0edee48d-5b66-4ca9-9fad-fc28df2200f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:12:27.964984 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1053/2100] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:12:27 2022] DELETE /identity/v3/projects/e74136856dcf42c48689e6976f16caad => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:12:36.458247 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=26 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:12:37.958613 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=70,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:12:42.463615 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36e847-eab8-44a9-ab52-fe5fae63bf43 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:42.464595 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36e847-eab8-44a9-ab52-fe5fae63bf43 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:42.464883 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36e847-eab8-44a9-ab52-fe5fae63bf43 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:42.466231 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1048/2101] 10.222.0.23 () {54 vars in 921 bytes} [Thu Oct 13 13:12:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:12:42.472984 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6d1d1-001d-4f68-8e37-c9cefa668052 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:42.473441 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6d1d1-001d-4f68-8e37-c9cefa668052 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:42.473544 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6d1d1-001d-4f68-8e37-c9cefa668052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:42.493630 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-aea6d1d1-001d-4f68-8e37-c9cefa668052 None None] MFA Rules not processed for user `12700d985a724595af630c81e63f86d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:12:42.494915 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-aea6d1d1-001d-4f68-8e37-c9cefa668052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:42.563364 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1054/2102] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Oct 13 13:12:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:12:42.575370 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36e847-eab8-44a9-ab52-fe5fae63bf43 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:42.577180 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-ff36e847-eab8-44a9-ab52-fe5fae63bf43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:42.635486 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36e847-eab8-44a9-ab52-fe5fae63bf43 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:42.638942 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b02c91-03e3-4c05-9e14-896cd07fdd1e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ce234c5b4643b2ac6341e0928f3830', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:42.639569 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b02c91-03e3-4c05-9e14-896cd07fdd1e service glance] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:42.639848 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b02c91-03e3-4c05-9e14-896cd07fdd1e service glance] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:42.640117 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b02c91-03e3-4c05-9e14-896cd07fdd1e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:42.652028 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b02c91-03e3-4c05-9e14-896cd07fdd1e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:42.653460 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b02c91-03e3-4c05-9e14-896cd07fdd1e service glance] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:42.664011 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1049/2103] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:12:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:12:42.734143 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82cabb-8bf1-44ee-8eb9-b81b09f7affc None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:42.734535 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82cabb-8bf1-44ee-8eb9-b81b09f7affc None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:42.734875 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82cabb-8bf1-44ee-8eb9-b81b09f7affc None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:42.736275 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1055/2104] 10.222.0.23 () {54 vars in 921 bytes} [Thu Oct 13 13:12:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:12:42.740885 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6122a-63b8-45b8-891d-b1ceed231c67 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:42.741173 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6122a-63b8-45b8-891d-b1ceed231c67 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:42.741456 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6122a-63b8-45b8-891d-b1ceed231c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:42.761169 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-36f6122a-63b8-45b8-891d-b1ceed231c67 None None] MFA Rules not processed for user `846251a661c04d3f9f195c4ac09b4189`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:12:42.762378 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-36f6122a-63b8-45b8-891d-b1ceed231c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:42.814668 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1050/2105] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Oct 13 13:12:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:12:42.827217 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca82cabb-8bf1-44ee-8eb9-b81b09f7affc None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:42.829173 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ca82cabb-8bf1-44ee-8eb9-b81b09f7affc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:42.843230 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca82cabb-8bf1-44ee-8eb9-b81b09f7affc None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:42.847016 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737904e3-b3d2-4ca6-93fc-21f2d0923909 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:42.847633 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737904e3-b3d2-4ca6-93fc-21f2d0923909 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:42.847914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737904e3-b3d2-4ca6-93fc-21f2d0923909 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:42.848186 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737904e3-b3d2-4ca6-93fc-21f2d0923909 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:42.855609 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737904e3-b3d2-4ca6-93fc-21f2d0923909 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:42.857025 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737904e3-b3d2-4ca6-93fc-21f2d0923909 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:42.866057 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1056/2106] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:12:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:12:42.956769 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa1824-b714-46ee-8857-5581a794c091 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:42.957320 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa1824-b714-46ee-8857-5581a794c091 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:42.957599 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa1824-b714-46ee-8857-5581a794c091 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:42.959324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1051/2107] 10.222.0.23 () {54 vars in 921 bytes} [Thu Oct 13 13:12:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:12:42.964922 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ad976-1a5e-4c16-a61f-32cbd37dcd19 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:42.965207 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ad976-1a5e-4c16-a61f-32cbd37dcd19 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:42.965479 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ad976-1a5e-4c16-a61f-32cbd37dcd19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:43.004686 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6d5ad976-1a5e-4c16-a61f-32cbd37dcd19 None None] MFA Rules not processed for user `12700d985a724595af630c81e63f86d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:12:43.005963 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6d5ad976-1a5e-4c16-a61f-32cbd37dcd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:43.016232 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1057/2108] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Oct 13 13:12:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:12:43.028170 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa1824-b714-46ee-8857-5581a794c091 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:43.041633 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa1824-b714-46ee-8857-5581a794c091 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:43.046317 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3b91c-7667-472f-88c2-d7d24ff253e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:43.046911 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3b91c-7667-472f-88c2-d7d24ff253e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:43.047219 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3b91c-7667-472f-88c2-d7d24ff253e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:43.047487 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3b91c-7667-472f-88c2-d7d24ff253e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:43.058792 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b3b91c-7667-472f-88c2-d7d24ff253e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:43.060250 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b3b91c-7667-472f-88c2-d7d24ff253e2 service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:43.073290 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1052/2109] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:12:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:12:47.517305 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5702ae-852c-42ca-9767-810841500d8d None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:47.517818 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5702ae-852c-42ca-9767-810841500d8d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:47.517933 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5702ae-852c-42ca-9767-810841500d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:47.538094 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-7b5702ae-852c-42ca-9767-810841500d8d None None] MFA Rules not processed for user `b6ae674f0efc445fa970fca64e13893a`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:12:47.539366 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7b5702ae-852c-42ca-9767-810841500d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:47.598415 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1058/2110] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Oct 13 13:12:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:12:47.610756 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3b91c-7667-472f-88c2-d7d24ff253e2 service neutron] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:47.612821 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-73b3b91c-7667-472f-88c2-d7d24ff253e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:47.692876 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3b91c-7667-472f-88c2-d7d24ff253e2 service neutron] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:47.698219 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485551e-b051-4ac7-9191-470d1fe3b009 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526000341fd146db8c661f1fd14d4801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:47.698839 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485551e-b051-4ac7-9191-470d1fe3b009 service swift] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:47.699140 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485551e-b051-4ac7-9191-470d1fe3b009 service swift] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:47.699411 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485551e-b051-4ac7-9191-470d1fe3b009 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:47.707859 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7485551e-b051-4ac7-9191-470d1fe3b009 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:47.709490 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7485551e-b051-4ac7-9191-470d1fe3b009 service swift] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:47.718229 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1053/2111] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 13 13:12:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 13 13:12:57.591879 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=65 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:12:57.715102 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=87 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:12:57.818649 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5702ae-852c-42ca-9767-810841500d8d None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:12:57.820199 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7b5702ae-852c-42ca-9767-810841500d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:57.900120 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5702ae-852c-42ca-9767-810841500d8d None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:12:57.910723 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6347c3cc-fa87-4670-9657-dc8c2af3cfbe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a971cdb7f9d4e6692c6326c103d7303', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:12:57.911554 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347c3cc-fa87-4670-9657-dc8c2af3cfbe service placement] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:12:57.911841 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347c3cc-fa87-4670-9657-dc8c2af3cfbe service placement] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:12:57.912113 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347c3cc-fa87-4670-9657-dc8c2af3cfbe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:12:57.913964 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6347c3cc-fa87-4670-9657-dc8c2af3cfbe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:12:57.931493 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6347c3cc-fa87-4670-9657-dc8c2af3cfbe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:12:57.933366 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6347c3cc-fa87-4670-9657-dc8c2af3cfbe service placement] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:12:57.945626 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1059/2112] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:12:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:13:07.939691 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=32 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:13:15.664950 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46ccae-234e-43af-b215-b21551c96ba4 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:13:15.665456 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46ccae-234e-43af-b215-b21551c96ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:13:15.665573 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46ccae-234e-43af-b215-b21551c96ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:13:15.685358 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-4d46ccae-234e-43af-b215-b21551c96ba4 None None] MFA Rules not processed for user `b6ae674f0efc445fa970fca64e13893a`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:13:15.686572 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-4d46ccae-234e-43af-b215-b21551c96ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:13:15.694139 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1054/2113] 10.222.0.23 () {58 vars in 967 bytes} [Thu Oct 13 13:13:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:13:15.716115 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6347c3cc-fa87-4670-9657-dc8c2af3cfbe service placement] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:13:15.727566 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6347c3cc-fa87-4670-9657-dc8c2af3cfbe service placement] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:13:15.732474 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004a0fc-f220-45f4-b24e-c97601d8fc72 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526000341fd146db8c661f1fd14d4801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:13:15.733069 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004a0fc-f220-45f4-b24e-c97601d8fc72 service swift] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:13:15.733348 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004a0fc-f220-45f4-b24e-c97601d8fc72 service swift] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:13:15.733620 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004a0fc-f220-45f4-b24e-c97601d8fc72 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:13:15.741405 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6004a0fc-f220-45f4-b24e-c97601d8fc72 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:13:15.742996 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6004a0fc-f220-45f4-b24e-c97601d8fc72 service swift] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:13:15.750781 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1060/2114] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 13 13:13:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 13 13:13:17.460002 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46ccae-234e-43af-b215-b21551c96ba4 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:13:17.469025 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46ccae-234e-43af-b215-b21551c96ba4 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:13:17.472515 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc24969-7942-465a-9b77-1771c2ba386b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:13:17.473108 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc24969-7942-465a-9b77-1771c2ba386b service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:13:17.473364 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc24969-7942-465a-9b77-1771c2ba386b service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:13:17.473632 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc24969-7942-465a-9b77-1771c2ba386b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:13:17.475377 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-9dc24969-7942-465a-9b77-1771c2ba386b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:13:17.538852 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc24969-7942-465a-9b77-1771c2ba386b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:13:17.540329 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc24969-7942-465a-9b77-1771c2ba386b service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:13:17.550929 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1055/2115] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:13:17 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:13:25.747638 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=6 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:13:27.545537 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=33 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:13:48.570558 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004a0fc-f220-45f4-b24e-c97601d8fc72 service swift] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:13:48.572310 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6004a0fc-f220-45f4-b24e-c97601d8fc72 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:13:48.637900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004a0fc-f220-45f4-b24e-c97601d8fc72 service swift] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:13:48.643036 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43067eb5-0300-4321-b640-16e64df8d705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:13:48.643685 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43067eb5-0300-4321-b640-16e64df8d705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:13:48.643995 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43067eb5-0300-4321-b640-16e64df8d705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:13:48.644235 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43067eb5-0300-4321-b640-16e64df8d705 admin admin] PATH_INFO: `/v3/users/8569afc405f24a0993a023c818a0c272` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:13:48.645355 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43067eb5-0300-4321-b640-16e64df8d705 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8569afc405f24a0993a023c818a0c272)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:13:48.647302 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43067eb5-0300-4321-b640-16e64df8d705 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:13:48.694581 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-43067eb5-0300-4321-b640-16e64df8d705 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8569afc405f24a0993a023c818a0c272'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:13:48.701639 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-43067eb5-0300-4321-b640-16e64df8d705 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8569afc405f24a0993a023c818a0c272'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:13:48.702257 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-43067eb5-0300-4321-b640-16e64df8d705 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8569afc405f24a0993a023c818a0c272'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:13:48.710465 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-43067eb5-0300-4321-b640-16e64df8d705 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8569afc405f24a0993a023c818a0c272'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:13:48.723024 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-43067eb5-0300-4321-b640-16e64df8d705 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8569afc405f24a0993a023c818a0c272'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:13:48.729483 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1061/2116] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:13:48 2022] DELETE /identity/v3/users/8569afc405f24a0993a023c818a0c272 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:13:48.927113 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc24969-7942-465a-9b77-1771c2ba386b service neutron] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:13:48.940481 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc24969-7942-465a-9b77-1771c2ba386b service neutron] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:13:48.996990 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4143acd-29e8-4430-9f95-7ae900ed5384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:13:48.997620 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4143acd-29e8-4430-9f95-7ae900ed5384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:13:48.997903 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4143acd-29e8-4430-9f95-7ae900ed5384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:13:48.998178 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4143acd-29e8-4430-9f95-7ae900ed5384 admin admin] PATH_INFO: `/v3/projects/f85144e13e254365a86aa785093c8575` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:13:48.999328 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4143acd-29e8-4430-9f95-7ae900ed5384 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f85144e13e254365a86aa785093c8575)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:13:49.001833 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4143acd-29e8-4430-9f95-7ae900ed5384 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:13:49.046395 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c4143acd-29e8-4430-9f95-7ae900ed5384 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f85144e13e254365a86aa785093c8575', 'request_id': 'req-c4143acd-29e8-4430-9f95-7ae900ed5384'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:13:49.052541 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c4143acd-29e8-4430-9f95-7ae900ed5384 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f85144e13e254365a86aa785093c8575', 'request_id': 'req-c4143acd-29e8-4430-9f95-7ae900ed5384'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:13:49.058490 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c4143acd-29e8-4430-9f95-7ae900ed5384 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f85144e13e254365a86aa785093c8575', 'request_id': 'req-c4143acd-29e8-4430-9f95-7ae900ed5384'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:13:49.070677 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c4143acd-29e8-4430-9f95-7ae900ed5384 admin admin] The token cache is being invalidate because project f85144e13e254365a86aa785093c8575 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:13:49.070963 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c4143acd-29e8-4430-9f95-7ae900ed5384 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:13:49.071616 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c4143acd-29e8-4430-9f95-7ae900ed5384 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:13:49.073065 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1056/2117] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:13:48 2022] DELETE /identity/v3/projects/f85144e13e254365a86aa785093c8575 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:13:58.725569 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:13:59.067792 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:14:05.345347 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029c6d8-9865-4ea6-8cad-95a604ceebf4 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:14:05.345881 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029c6d8-9865-4ea6-8cad-95a604ceebf4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:14:05.345881 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029c6d8-9865-4ea6-8cad-95a604ceebf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:14:05.365516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-e029c6d8-9865-4ea6-8cad-95a604ceebf4 None None] MFA Rules not processed for user `454203f47b434cca952a1cef56de559f`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:14:05.366731 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e029c6d8-9865-4ea6-8cad-95a604ceebf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:14:05.420954 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1062/2118] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:14:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:14:07.873041 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf518d-8945-4784-90a4-38a25f3dc60a None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:14:07.873534 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf518d-8945-4784-90a4-38a25f3dc60a None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:14:07.873648 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf518d-8945-4784-90a4-38a25f3dc60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:14:07.892885 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-e7bf518d-8945-4784-90a4-38a25f3dc60a None None] MFA Rules not processed for user `b5dbaf53464b44fe8d9e16f5013587b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:14:07.894076 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-e7bf518d-8945-4784-90a4-38a25f3dc60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:14:07.951914 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1057/2119] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:14:07 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:14:15.413972 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=25 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:14:17.945186 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=25 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:14:23.462471 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a341bc4-1f44-4003-a992-1a2da80aa94e None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:14:23.462974 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a341bc4-1f44-4003-a992-1a2da80aa94e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:14:23.463104 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a341bc4-1f44-4003-a992-1a2da80aa94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:14:23.481799 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-4a341bc4-1f44-4003-a992-1a2da80aa94e None None] MFA Rules not processed for user `2e783caddcad473dbdd764919286e244`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:14:23.483002 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4a341bc4-1f44-4003-a992-1a2da80aa94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:14:23.536097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1063/2120] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:14:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:14:33.529511 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=25 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:14:37.143868 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0238d-2c41-4047-9495-4d55c941d730 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:14:37.144330 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0238d-2c41-4047-9495-4d55c941d730 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:14:37.144453 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0238d-2c41-4047-9495-4d55c941d730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:14:37.164144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-8cb0238d-2c41-4047-9495-4d55c941d730 None None] MFA Rules not processed for user `b6ae674f0efc445fa970fca64e13893a`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:14:37.165340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-8cb0238d-2c41-4047-9495-4d55c941d730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:14:37.226357 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1058/2121] 10.222.0.23 () {58 vars in 967 bytes} [Thu Oct 13 13:14:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:14:37.276277 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a341bc4-1f44-4003-a992-1a2da80aa94e None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:14:37.277720 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4a341bc4-1f44-4003-a992-1a2da80aa94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:14:37.352926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a341bc4-1f44-4003-a992-1a2da80aa94e None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:14:37.357746 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b0fab-52d2-4410-8d97-c000f51a342b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526000341fd146db8c661f1fd14d4801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:14:37.358372 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b0fab-52d2-4410-8d97-c000f51a342b service swift] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:14:37.358649 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b0fab-52d2-4410-8d97-c000f51a342b service swift] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:14:37.359038 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b0fab-52d2-4410-8d97-c000f51a342b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:14:37.367267 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b0fab-52d2-4410-8d97-c000f51a342b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:14:37.368939 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b0fab-52d2-4410-8d97-c000f51a342b service swift] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:14:37.377484 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1064/2122] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 13 13:14:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 13 13:14:43.874873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb0238d-2c41-4047-9495-4d55c941d730 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:14:43.876763 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-8cb0238d-2c41-4047-9495-4d55c941d730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:14:43.946075 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb0238d-2c41-4047-9495-4d55c941d730 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:14:43.950671 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f0d59-2b5b-48a7-a592-96ddc90d654e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:14:43.951860 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f0d59-2b5b-48a7-a592-96ddc90d654e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:14:43.952139 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f0d59-2b5b-48a7-a592-96ddc90d654e admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:14:43.952409 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f0d59-2b5b-48a7-a592-96ddc90d654e admin admin] PATH_INFO: `/v3/users/7d89667e52b24295ba619203c02c7bd4` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:14:43.953549 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234f0d59-2b5b-48a7-a592-96ddc90d654e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d89667e52b24295ba619203c02c7bd4)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:14:43.955517 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234f0d59-2b5b-48a7-a592-96ddc90d654e admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:14:44.001152 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-234f0d59-2b5b-48a7-a592-96ddc90d654e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d89667e52b24295ba619203c02c7bd4'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:14:44.008624 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-234f0d59-2b5b-48a7-a592-96ddc90d654e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d89667e52b24295ba619203c02c7bd4'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:14:44.009301 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-234f0d59-2b5b-48a7-a592-96ddc90d654e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d89667e52b24295ba619203c02c7bd4'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:14:44.010426 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-234f0d59-2b5b-48a7-a592-96ddc90d654e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d89667e52b24295ba619203c02c7bd4'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:14:44.020061 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-234f0d59-2b5b-48a7-a592-96ddc90d654e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d89667e52b24295ba619203c02c7bd4'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:14:44.041425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1059/2123] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:14:43 2022] DELETE /identity/v3/users/7d89667e52b24295ba619203c02c7bd4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:14:44.213006 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b0fab-52d2-4410-8d97-c000f51a342b service swift] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:14:44.223474 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b0fab-52d2-4410-8d97-c000f51a342b service swift] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:14:44.280180 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda14ad-5b3a-4527-8d7f-0e018e5fef90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:14:44.280730 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda14ad-5b3a-4527-8d7f-0e018e5fef90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:14:44.281015 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda14ad-5b3a-4527-8d7f-0e018e5fef90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:14:44.281286 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda14ad-5b3a-4527-8d7f-0e018e5fef90 admin admin] PATH_INFO: `/v3/projects/3ed5038c799342d48edc980983f1d996` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:14:44.282412 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cda14ad-5b3a-4527-8d7f-0e018e5fef90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ed5038c799342d48edc980983f1d996)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:14:44.284351 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cda14ad-5b3a-4527-8d7f-0e018e5fef90 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:14:44.329248 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0cda14ad-5b3a-4527-8d7f-0e018e5fef90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ed5038c799342d48edc980983f1d996', 'request_id': 'req-0cda14ad-5b3a-4527-8d7f-0e018e5fef90'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:14:44.334626 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0cda14ad-5b3a-4527-8d7f-0e018e5fef90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ed5038c799342d48edc980983f1d996', 'request_id': 'req-0cda14ad-5b3a-4527-8d7f-0e018e5fef90'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:14:44.340892 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0cda14ad-5b3a-4527-8d7f-0e018e5fef90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ed5038c799342d48edc980983f1d996', 'request_id': 'req-0cda14ad-5b3a-4527-8d7f-0e018e5fef90'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:14:44.353521 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0cda14ad-5b3a-4527-8d7f-0e018e5fef90 admin admin] The token cache is being invalidate because project 3ed5038c799342d48edc980983f1d996 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:14:44.353806 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0cda14ad-5b3a-4527-8d7f-0e018e5fef90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:14:44.354411 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0cda14ad-5b3a-4527-8d7f-0e018e5fef90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:14:44.356299 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1065/2124] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:14:44 2022] DELETE /identity/v3/projects/3ed5038c799342d48edc980983f1d996 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:14:44.360486 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f0d59-2b5b-48a7-a592-96ddc90d654e admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:14:44.361883 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-234f0d59-2b5b-48a7-a592-96ddc90d654e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:14:44.375813 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f0d59-2b5b-48a7-a592-96ddc90d654e admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:14:44.395807 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:14:44.436779 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb6aa7-197a-485f-9f8d-a903c9a23bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:14:44.437465 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb6aa7-197a-485f-9f8d-a903c9a23bd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:14:44.437749 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb6aa7-197a-485f-9f8d-a903c9a23bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:14:44.438027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb6aa7-197a-485f-9f8d-a903c9a23bd7 admin admin] PATH_INFO: `/v3/users/afa565808a2d4ea9b6274cabcf1f4572` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:14:44.439202 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb6aa7-197a-485f-9f8d-a903c9a23bd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afa565808a2d4ea9b6274cabcf1f4572)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:14:44.441134 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb6aa7-197a-485f-9f8d-a903c9a23bd7 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:14:44.487401 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-74fb6aa7-197a-485f-9f8d-a903c9a23bd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afa565808a2d4ea9b6274cabcf1f4572'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:14:44.496120 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-74fb6aa7-197a-485f-9f8d-a903c9a23bd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afa565808a2d4ea9b6274cabcf1f4572'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:14:44.496864 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-74fb6aa7-197a-485f-9f8d-a903c9a23bd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afa565808a2d4ea9b6274cabcf1f4572'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:14:44.497522 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-74fb6aa7-197a-485f-9f8d-a903c9a23bd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afa565808a2d4ea9b6274cabcf1f4572'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:14:44.506050 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-74fb6aa7-197a-485f-9f8d-a903c9a23bd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afa565808a2d4ea9b6274cabcf1f4572'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:14:44.524302 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1060/2125] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:14:44 2022] DELETE /identity/v3/users/afa565808a2d4ea9b6274cabcf1f4572 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:14:44.626037 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda14ad-5b3a-4527-8d7f-0e018e5fef90 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:14:44.627497 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0cda14ad-5b3a-4527-8d7f-0e018e5fef90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:14:44.649800 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:14:44.691601 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda14ad-5b3a-4527-8d7f-0e018e5fef90 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:14:44.705550 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb6aa7-197a-485f-9f8d-a903c9a23bd7 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:14:44.707587 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-74fb6aa7-197a-485f-9f8d-a903c9a23bd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:14:44.748780 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2887e21-ed4b-4b28-9855-80c9bda8cd83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:14:44.749393 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2887e21-ed4b-4b28-9855-80c9bda8cd83 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:14:44.749673 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2887e21-ed4b-4b28-9855-80c9bda8cd83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:14:44.749945 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2887e21-ed4b-4b28-9855-80c9bda8cd83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:14:44.751751 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d2887e21-ed4b-4b28-9855-80c9bda8cd83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:14:44.778398 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb6aa7-197a-485f-9f8d-a903c9a23bd7 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:14:44.783520 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f60d6-df86-45ed-8cf9-3605113656bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:14:44.784151 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f60d6-df86-45ed-8cf9-3605113656bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:14:44.784546 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f60d6-df86-45ed-8cf9-3605113656bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:14:44.784825 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f60d6-df86-45ed-8cf9-3605113656bb admin admin] PATH_INFO: `/v3/projects/e5527143235a4fab8d08d7ef484670ce` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:14:44.786031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f60d6-df86-45ed-8cf9-3605113656bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5527143235a4fab8d08d7ef484670ce)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:14:44.788032 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f60d6-df86-45ed-8cf9-3605113656bb admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:14:44.823356 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2887e21-ed4b-4b28-9855-80c9bda8cd83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:14:44.824820 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2887e21-ed4b-4b28-9855-80c9bda8cd83 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:14:44.834033 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:14:44.836260 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1066/2126] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:14:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:14:44.837743 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-943f60d6-df86-45ed-8cf9-3605113656bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5527143235a4fab8d08d7ef484670ce', 'request_id': 'req-943f60d6-df86-45ed-8cf9-3605113656bb'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:14:44.844869 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-943f60d6-df86-45ed-8cf9-3605113656bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5527143235a4fab8d08d7ef484670ce', 'request_id': 'req-943f60d6-df86-45ed-8cf9-3605113656bb'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:14:44.851351 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-943f60d6-df86-45ed-8cf9-3605113656bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5527143235a4fab8d08d7ef484670ce', 'request_id': 'req-943f60d6-df86-45ed-8cf9-3605113656bb'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:14:44.865057 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-943f60d6-df86-45ed-8cf9-3605113656bb admin admin] The token cache is being invalidate because project e5527143235a4fab8d08d7ef484670ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:14:44.865486 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-943f60d6-df86-45ed-8cf9-3605113656bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:14:44.866127 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-943f60d6-df86-45ed-8cf9-3605113656bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:14:44.867679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1061/2127] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:14:44 2022] DELETE /identity/v3/projects/e5527143235a4fab8d08d7ef484670ce => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:14:44.872604 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333265f-3ade-499b-a69d-b27b1c24f940 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:14:44.873632 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333265f-3ade-499b-a69d-b27b1c24f940 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:14:44.873940 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333265f-3ade-499b-a69d-b27b1c24f940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:14:44.896566 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-9333265f-3ade-499b-a69d-b27b1c24f940 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:14:44.897960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9333265f-3ade-499b-a69d-b27b1c24f940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:14:44.977702 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1067/2128] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:14:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:14:44.986081 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717aa1e0-d75b-431b-b119-6087553f2598 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:14:44.986590 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717aa1e0-d75b-431b-b119-6087553f2598 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:14:44.986700 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717aa1e0-d75b-431b-b119-6087553f2598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:14:45.015801 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-717aa1e0-d75b-431b-b119-6087553f2598 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:14:45.017023 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-717aa1e0-d75b-431b-b119-6087553f2598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:14:45.025162 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1062/2129] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:14:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:14:45.029967 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9333265f-3ade-499b-a69d-b27b1c24f940 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:14:45.039559 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9333265f-3ade-499b-a69d-b27b1c24f940 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:14:45.044380 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d31a72-aa8e-4600-a3b8-587f894b9b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:14:45.045624 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d31a72-aa8e-4600-a3b8-587f894b9b12 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:14:45.046061 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d31a72-aa8e-4600-a3b8-587f894b9b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:14:45.046408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d31a72-aa8e-4600-a3b8-587f894b9b12 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:14:45.047279 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d31a72-aa8e-4600-a3b8-587f894b9b12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:14:45.047613 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d31a72-aa8e-4600-a3b8-587f894b9b12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:14:45.049140 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d31a72-aa8e-4600-a3b8-587f894b9b12 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:14:45.061854 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1068/2130] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 13:14:45.066315 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717aa1e0-d75b-431b-b119-6087553f2598 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:14:45.075561 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717aa1e0-d75b-431b-b119-6087553f2598 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:14:45.080214 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066956db-1292-48d3-9d11-c62f7ae2d304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:14:45.080806 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066956db-1292-48d3-9d11-c62f7ae2d304 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:14:45.081084 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066956db-1292-48d3-9d11-c62f7ae2d304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:14:45.081353 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066956db-1292-48d3-9d11-c62f7ae2d304 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:14:45.081993 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066956db-1292-48d3-9d11-c62f7ae2d304 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:14:45.083936 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066956db-1292-48d3-9d11-c62f7ae2d304 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:14:45.114498 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1063/2131] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:14:45 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:14:45.115607 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d31a72-aa8e-4600-a3b8-587f894b9b12 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:14:45.122241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:14:45.126347 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d31a72-aa8e-4600-a3b8-587f894b9b12 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:14:45.181769 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b431b30-5f4a-4fee-85c8-80feeea5f28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:14:45.182366 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b431b30-5f4a-4fee-85c8-80feeea5f28d admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:14:45.182647 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b431b30-5f4a-4fee-85c8-80feeea5f28d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:14:45.182918 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b431b30-5f4a-4fee-85c8-80feeea5f28d admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:14:45.183587 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b431b30-5f4a-4fee-85c8-80feeea5f28d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:14:45.185493 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b431b30-5f4a-4fee-85c8-80feeea5f28d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:14:45.206257 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1069/2132] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:14:45 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:14:45.215746 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066956db-1292-48d3-9d11-c62f7ae2d304 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:14:45.226478 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066956db-1292-48d3-9d11-c62f7ae2d304 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:14:45.230929 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c4404-fb17-460f-ac2f-859107f2c008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:14:45.231528 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c4404-fb17-460f-ac2f-859107f2c008 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:14:45.231804 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c4404-fb17-460f-ac2f-859107f2c008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:14:45.232071 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c4404-fb17-460f-ac2f-859107f2c008 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:14:45.232699 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c4404-fb17-460f-ac2f-859107f2c008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:14:45.234282 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c4404-fb17-460f-ac2f-859107f2c008 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:14:45.243849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1064/2133] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:14:45.247721 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b431b30-5f4a-4fee-85c8-80feeea5f28d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:14:45.257575 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b431b30-5f4a-4fee-85c8-80feeea5f28d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:14:45.262148 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd197b3-8310-42a9-946a-d04f1985637d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:14:45.262738 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd197b3-8310-42a9-946a-d04f1985637d admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:14:45.263020 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd197b3-8310-42a9-946a-d04f1985637d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:14:45.263322 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd197b3-8310-42a9-946a-d04f1985637d admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:14:45.264516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd197b3-8310-42a9-946a-d04f1985637d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:14:45.266095 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd197b3-8310-42a9-946a-d04f1985637d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:14:45.276663 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1070/2134] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:14:45.281057 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c4404-fb17-460f-ac2f-859107f2c008 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:14:45.290927 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c4404-fb17-460f-ac2f-859107f2c008 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:14:45.295567 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3351c5-b38b-4206-9dd3-d9679937bb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:14:45.296209 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3351c5-b38b-4206-9dd3-d9679937bb01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:14:45.296492 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3351c5-b38b-4206-9dd3-d9679937bb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:14:45.296768 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3351c5-b38b-4206-9dd3-d9679937bb01 admin admin] PATH_INFO: `/v3/projects/aff81c38c17b4e82864854ceac6587df/users/6723d3c8aa144452b46564649cf38ee3/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:14:45.311282 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3351c5-b38b-4206-9dd3-d9679937bb01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aff81c38c17b4e82864854ceac6587df, user_id=6723d3c8aa144452b46564649cf38ee3, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:14:45.313959 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3351c5-b38b-4206-9dd3-d9679937bb01 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:14:45.327370 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1065/2135] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:14:45 2022] PUT /identity/v3/projects/aff81c38c17b4e82864854ceac6587df/users/6723d3c8aa144452b46564649cf38ee3/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:14:45.332189 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddd610-b72f-4036-a20d-5ebe5b3e81f7 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:14:45.332510 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddd610-b72f-4036-a20d-5ebe5b3e81f7 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:14:45.332786 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddd610-b72f-4036-a20d-5ebe5b3e81f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:14:45.351734 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-bfddd610-b72f-4036-a20d-5ebe5b3e81f7 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:14:45.352932 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-bfddd610-b72f-4036-a20d-5ebe5b3e81f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:14:45.409542 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1071/2136] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:14:45.419419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3351c5-b38b-4206-9dd3-d9679937bb01 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:14:45.421119 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-5d3351c5-b38b-4206-9dd3-d9679937bb01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:14:45.477620 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3351c5-b38b-4206-9dd3-d9679937bb01 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:14:45.481212 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca92a27-0bc5-40b6-b9ae-bb202b0a2ca5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:14:45.481814 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca92a27-0bc5-40b6-b9ae-bb202b0a2ca5 service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:14:45.482095 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca92a27-0bc5-40b6-b9ae-bb202b0a2ca5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:14:45.482368 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca92a27-0bc5-40b6-b9ae-bb202b0a2ca5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:14:45.491160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca92a27-0bc5-40b6-b9ae-bb202b0a2ca5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:14:45.492588 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca92a27-0bc5-40b6-b9ae-bb202b0a2ca5 service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:14:45.502257 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1066/2137] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:14:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:14:46.044437 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfddd610-b72f-4036-a20d-5ebe5b3e81f7 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:14:46.046462 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-bfddd610-b72f-4036-a20d-5ebe5b3e81f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:14:46.058778 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfddd610-b72f-4036-a20d-5ebe5b3e81f7 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:14:46.063695 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252082f6-1654-408a-a972-063c0b9d52bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:14:46.064302 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252082f6-1654-408a-a972-063c0b9d52bd service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:14:46.064583 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252082f6-1654-408a-a972-063c0b9d52bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:14:46.064860 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252082f6-1654-408a-a972-063c0b9d52bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:14:46.066565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-252082f6-1654-408a-a972-063c0b9d52bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:14:46.146482 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252082f6-1654-408a-a972-063c0b9d52bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:14:46.147977 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252082f6-1654-408a-a972-063c0b9d52bd service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:14:46.160076 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1072/2138] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:14:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:14:50.271026 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69569b6b-b698-4c4e-8bea-759be7cda572 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:14:50.271574 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69569b6b-b698-4c4e-8bea-759be7cda572 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:14:50.271677 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69569b6b-b698-4c4e-8bea-759be7cda572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:14:50.300646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-69569b6b-b698-4c4e-8bea-759be7cda572 None None] MFA Rules not processed for user `6723d3c8aa144452b46564649cf38ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:14:50.301936 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-69569b6b-b698-4c4e-8bea-759be7cda572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:14:50.367832 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1067/2139] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:14:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:14:50.372313 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252082f6-1654-408a-a972-063c0b9d52bd service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:14:50.383031 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252082f6-1654-408a-a972-063c0b9d52bd service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:14:50.387535 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db343cc3-dec4-47c2-aac6-4bf7054f4357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:14:50.388150 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db343cc3-dec4-47c2-aac6-4bf7054f4357 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:14:50.388413 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db343cc3-dec4-47c2-aac6-4bf7054f4357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:14:50.388764 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db343cc3-dec4-47c2-aac6-4bf7054f4357 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:14:50.389435 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db343cc3-dec4-47c2-aac6-4bf7054f4357 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:14:50.391488 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db343cc3-dec4-47c2-aac6-4bf7054f4357 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:14:50.397329 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:14:50.429882 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1073/2140] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:14:50 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:14:50.431043 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69569b6b-b698-4c4e-8bea-759be7cda572 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:14:50.437388 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:14:50.443801 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69569b6b-b698-4c4e-8bea-759be7cda572 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:14:50.510852 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d1e27-3f98-46e7-96e9-40bdcce4016b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:14:50.511656 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d1e27-3f98-46e7-96e9-40bdcce4016b admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:14:50.511996 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d1e27-3f98-46e7-96e9-40bdcce4016b admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:14:50.512312 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d1e27-3f98-46e7-96e9-40bdcce4016b admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:14:50.512991 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221d1e27-3f98-46e7-96e9-40bdcce4016b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:14:50.515771 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221d1e27-3f98-46e7-96e9-40bdcce4016b admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:14:50.550022 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1068/2141] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:14:50 2022] POST /identity/v3/users => generated 292 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:14:50.551319 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db343cc3-dec4-47c2-aac6-4bf7054f4357 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:14:50.562105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db343cc3-dec4-47c2-aac6-4bf7054f4357 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:14:50.572757 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c17c0-cf82-46c5-a056-5b96deb42251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:14:50.573608 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c17c0-cf82-46c5-a056-5b96deb42251 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:14:50.573986 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c17c0-cf82-46c5-a056-5b96deb42251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:14:50.574410 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c17c0-cf82-46c5-a056-5b96deb42251 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:14:50.575340 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c17c0-cf82-46c5-a056-5b96deb42251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:14:50.578444 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c17c0-cf82-46c5-a056-5b96deb42251 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:14:50.592378 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1074/2142] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:14:50.596856 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d1e27-3f98-46e7-96e9-40bdcce4016b admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:14:50.608390 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d1e27-3f98-46e7-96e9-40bdcce4016b admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:14:50.613155 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b85f3-a73e-4a9c-8bf0-a01871033fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:14:50.613934 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b85f3-a73e-4a9c-8bf0-a01871033fde admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:14:50.614234 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b85f3-a73e-4a9c-8bf0-a01871033fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:14:50.614543 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b85f3-a73e-4a9c-8bf0-a01871033fde admin admin] PATH_INFO: `/v3/projects/cdb20a29152744f580ec016b985892d5/users/64d0f55920a54472828632465d053d66/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:14:50.633096 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b85f3-a73e-4a9c-8bf0-a01871033fde admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdb20a29152744f580ec016b985892d5, user_id=64d0f55920a54472828632465d053d66, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:14:50.635873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b85f3-a73e-4a9c-8bf0-a01871033fde admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:14:50.651246 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1069/2143] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:14:50 2022] PUT /identity/v3/projects/cdb20a29152744f580ec016b985892d5/users/64d0f55920a54472828632465d053d66/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:14:50.663553 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c17c0-cf82-46c5-a056-5b96deb42251 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:14:50.675266 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c17c0-cf82-46c5-a056-5b96deb42251 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:14:50.733359 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c69c22-4284-4c26-994d-a5806b501219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:14:50.733967 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c69c22-4284-4c26-994d-a5806b501219 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:14:50.734250 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c69c22-4284-4c26-994d-a5806b501219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:14:50.734525 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c69c22-4284-4c26-994d-a5806b501219 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:14:50.735329 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c69c22-4284-4c26-994d-a5806b501219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:14:50.736995 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c69c22-4284-4c26-994d-a5806b501219 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:14:50.751441 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1075/2144] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:14:50.752610 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b85f3-a73e-4a9c-8bf0-a01871033fde admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:14:50.764251 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b85f3-a73e-4a9c-8bf0-a01871033fde admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:14:50.768907 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188f328-80cd-4832-a3c9-8c09210e2904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:14:50.769559 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188f328-80cd-4832-a3c9-8c09210e2904 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:14:50.769832 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188f328-80cd-4832-a3c9-8c09210e2904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:14:50.770123 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188f328-80cd-4832-a3c9-8c09210e2904 admin admin] PATH_INFO: `/v3/projects/cdb20a29152744f580ec016b985892d5/users/64d0f55920a54472828632465d053d66/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:14:50.772079 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f188f328-80cd-4832-a3c9-8c09210e2904 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdb20a29152744f580ec016b985892d5, user_id=64d0f55920a54472828632465d053d66, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:14:50.774716 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f188f328-80cd-4832-a3c9-8c09210e2904 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:14:50.785288 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1070/2145] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:14:50 2022] PUT /identity/v3/projects/cdb20a29152744f580ec016b985892d5/users/64d0f55920a54472828632465d053d66/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:14:55.804218 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a82a0c-3f94-4256-b970-d38a9a4a0f74 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:14:55.804729 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a82a0c-3f94-4256-b970-d38a9a4a0f74 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:14:55.804834 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a82a0c-3f94-4256-b970-d38a9a4a0f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:14:55.847090 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-52a82a0c-3f94-4256-b970-d38a9a4a0f74 None None] MFA Rules not processed for user `64d0f55920a54472828632465d053d66`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:14:55.847090 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-52a82a0c-3f94-4256-b970-d38a9a4a0f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:14:55.919955 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1076/2146] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:14:55 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:14:55.934562 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188f328-80cd-4832-a3c9-8c09210e2904 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:14:55.936483 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-f188f328-80cd-4832-a3c9-8c09210e2904 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:14:56.013077 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188f328-80cd-4832-a3c9-8c09210e2904 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:14:56.019101 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be28cda-b265-46d9-9082-a6faa5a682cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:14:56.019849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be28cda-b265-46d9-9082-a6faa5a682cb service nova] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:14:56.020129 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be28cda-b265-46d9-9082-a6faa5a682cb service nova] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:14:56.020399 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be28cda-b265-46d9-9082-a6faa5a682cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:14:56.070162 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:14:56.081149 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be28cda-b265-46d9-9082-a6faa5a682cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:14:56.082914 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be28cda-b265-46d9-9082-a6faa5a682cb service nova] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:14:56.092757 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1071/2147] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:14:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:14:57.362233 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457c419-5fc9-4db4-a9f9-55f7785502ec None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:14:57.363187 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457c419-5fc9-4db4-a9f9-55f7785502ec None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:14:57.363511 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3457c419-5fc9-4db4-a9f9-55f7785502ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:14:57.385121 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-3457c419-5fc9-4db4-a9f9-55f7785502ec None None] MFA Rules not processed for user `6723d3c8aa144452b46564649cf38ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:14:57.386467 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3457c419-5fc9-4db4-a9f9-55f7785502ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:14:57.399220 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1077/2148] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:14:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:15:04.251369 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be28cda-b265-46d9-9082-a6faa5a682cb service nova] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:04.305350 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be28cda-b265-46d9-9082-a6faa5a682cb service nova] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:04.309389 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c909b-3b58-47c7-a593-24f2d152bc8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:04.310810 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c909b-3b58-47c7-a593-24f2d152bc8b service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:04.311267 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c909b-3b58-47c7-a593-24f2d152bc8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:04.311719 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374c909b-3b58-47c7-a593-24f2d152bc8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:04.313529 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-374c909b-3b58-47c7-a593-24f2d152bc8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:04.374765 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c909b-3b58-47c7-a593-24f2d152bc8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:04.376248 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374c909b-3b58-47c7-a593-24f2d152bc8b service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:04.386046 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1072/2149] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:15:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:15:07.290839 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3457c419-5fc9-4db4-a9f9-55f7785502ec None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:07.293474 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3457c419-5fc9-4db4-a9f9-55f7785502ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:07.313645 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3457c419-5fc9-4db4-a9f9-55f7785502ec None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:07.319839 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcca00e-6a1a-44ef-82bc-da515a110e38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:07.320612 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcca00e-6a1a-44ef-82bc-da515a110e38 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:07.320990 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcca00e-6a1a-44ef-82bc-da515a110e38 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:07.321360 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcca00e-6a1a-44ef-82bc-da515a110e38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:07.323343 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6fcca00e-6a1a-44ef-82bc-da515a110e38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:07.372005 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:15:07.383680 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcca00e-6a1a-44ef-82bc-da515a110e38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:07.385736 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcca00e-6a1a-44ef-82bc-da515a110e38 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:07.395893 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1078/2150] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:15:07 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:15:07.850895 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c909b-3b58-47c7-a593-24f2d152bc8b service neutron] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:07.852499 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-374c909b-3b58-47c7-a593-24f2d152bc8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:07.866493 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374c909b-3b58-47c7-a593-24f2d152bc8b service neutron] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:07.871023 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7130b5-3868-4d10-b3d0-c65184dc544c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:07.871667 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7130b5-3868-4d10-b3d0-c65184dc544c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:07.871941 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7130b5-3868-4d10-b3d0-c65184dc544c admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:07.872810 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7130b5-3868-4d10-b3d0-c65184dc544c admin admin] PATH_INFO: `/v3/users/454203f47b434cca952a1cef56de559f` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:07.873925 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7130b5-3868-4d10-b3d0-c65184dc544c admin admin] RBAC: Authorizing `identity:delete_user(user_id=454203f47b434cca952a1cef56de559f)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:07.875876 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7130b5-3868-4d10-b3d0-c65184dc544c admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:07.921671 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-9e7130b5-3868-4d10-b3d0-c65184dc544c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '454203f47b434cca952a1cef56de559f'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:07.928757 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-9e7130b5-3868-4d10-b3d0-c65184dc544c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '454203f47b434cca952a1cef56de559f'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:07.929432 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-9e7130b5-3868-4d10-b3d0-c65184dc544c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '454203f47b434cca952a1cef56de559f'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:07.930036 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-9e7130b5-3868-4d10-b3d0-c65184dc544c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '454203f47b434cca952a1cef56de559f'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:07.937883 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-9e7130b5-3868-4d10-b3d0-c65184dc544c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '454203f47b434cca952a1cef56de559f'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:07.957954 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1073/2151] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:15:07 2022] DELETE /identity/v3/users/454203f47b434cca952a1cef56de559f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:15:08.132667 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcca00e-6a1a-44ef-82bc-da515a110e38 service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:08.144598 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcca00e-6a1a-44ef-82bc-da515a110e38 service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:08.202405 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da7aef8-8585-4c45-a267-e0a1006b8859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:08.203006 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7aef8-8585-4c45-a267-e0a1006b8859 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:08.203386 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7aef8-8585-4c45-a267-e0a1006b8859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:08.203655 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7aef8-8585-4c45-a267-e0a1006b8859 admin admin] PATH_INFO: `/v3/projects/5e7312d0ac9248f2bfbece5611913ab2` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:08.204776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da7aef8-8585-4c45-a267-e0a1006b8859 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e7312d0ac9248f2bfbece5611913ab2)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:08.206849 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da7aef8-8585-4c45-a267-e0a1006b8859 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:08.253710 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1da7aef8-8585-4c45-a267-e0a1006b8859 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e7312d0ac9248f2bfbece5611913ab2', 'request_id': 'req-1da7aef8-8585-4c45-a267-e0a1006b8859'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:08.259549 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1da7aef8-8585-4c45-a267-e0a1006b8859 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e7312d0ac9248f2bfbece5611913ab2', 'request_id': 'req-1da7aef8-8585-4c45-a267-e0a1006b8859'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:08.265565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1da7aef8-8585-4c45-a267-e0a1006b8859 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e7312d0ac9248f2bfbece5611913ab2', 'request_id': 'req-1da7aef8-8585-4c45-a267-e0a1006b8859'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:08.277562 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1da7aef8-8585-4c45-a267-e0a1006b8859 admin admin] The token cache is being invalidate because project 5e7312d0ac9248f2bfbece5611913ab2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:15:08.277844 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1da7aef8-8585-4c45-a267-e0a1006b8859 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:08.279041 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1da7aef8-8585-4c45-a267-e0a1006b8859 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:08.280646 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1079/2152] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:15:08 2022] DELETE /identity/v3/projects/5e7312d0ac9248f2bfbece5611913ab2 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:15:08.285655 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55d7ae-a351-41fa-b962-74c105759244 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:08.285944 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55d7ae-a351-41fa-b962-74c105759244 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:08.286221 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55d7ae-a351-41fa-b962-74c105759244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:08.308235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-fc55d7ae-a351-41fa-b962-74c105759244 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:15:08.309432 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-fc55d7ae-a351-41fa-b962-74c105759244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:08.317547 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:15:08.369714 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1074/2153] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:15:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:15:08.377704 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5905d-53d5-4e0f-8afa-15f7011bd42b None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:08.378160 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5905d-53d5-4e0f-8afa-15f7011bd42b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:08.378272 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5905d-53d5-4e0f-8afa-15f7011bd42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:08.397950 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f5d5905d-53d5-4e0f-8afa-15f7011bd42b None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:15:08.399187 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f5d5905d-53d5-4e0f-8afa-15f7011bd42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:08.407641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1080/2154] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:15:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:15:08.411843 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55d7ae-a351-41fa-b962-74c105759244 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:08.421371 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55d7ae-a351-41fa-b962-74c105759244 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:08.426185 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dfcd04-b327-4a81-9c6a-b3aceecbf2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:08.426776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dfcd04-b327-4a81-9c6a-b3aceecbf2cb admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:08.427071 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dfcd04-b327-4a81-9c6a-b3aceecbf2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:08.427349 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dfcd04-b327-4a81-9c6a-b3aceecbf2cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:08.427991 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dfcd04-b327-4a81-9c6a-b3aceecbf2cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:15:08.428291 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dfcd04-b327-4a81-9c6a-b3aceecbf2cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:08.429875 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dfcd04-b327-4a81-9c6a-b3aceecbf2cb admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:08.440980 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1075/2155] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:15:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:15:08.445321 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5905d-53d5-4e0f-8afa-15f7011bd42b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:08.454961 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5905d-53d5-4e0f-8afa-15f7011bd42b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:08.459559 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6099cc-8ccc-48ed-9964-6eed129bc04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:08.460151 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6099cc-8ccc-48ed-9964-6eed129bc04b admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:08.460432 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6099cc-8ccc-48ed-9964-6eed129bc04b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:08.460707 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6099cc-8ccc-48ed-9964-6eed129bc04b admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:08.461339 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6099cc-8ccc-48ed-9964-6eed129bc04b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:08.463260 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6099cc-8ccc-48ed-9964-6eed129bc04b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:08.489938 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1081/2156] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:15:08 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:15:08.494038 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dfcd04-b327-4a81-9c6a-b3aceecbf2cb admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:08.504494 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dfcd04-b327-4a81-9c6a-b3aceecbf2cb admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:08.561216 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3034de-65da-40ad-adbf-80b82fab577d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:08.561816 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3034de-65da-40ad-adbf-80b82fab577d admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:08.562094 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3034de-65da-40ad-adbf-80b82fab577d admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:08.562365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3034de-65da-40ad-adbf-80b82fab577d admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:08.562996 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3034de-65da-40ad-adbf-80b82fab577d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:08.564951 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3034de-65da-40ad-adbf-80b82fab577d admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:08.586489 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1076/2157] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:15:08 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:15:08.590194 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6099cc-8ccc-48ed-9964-6eed129bc04b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:08.600963 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6099cc-8ccc-48ed-9964-6eed129bc04b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:08.605585 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ca685-6892-4617-ab64-a51fedcb1d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:08.606182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ca685-6892-4617-ab64-a51fedcb1d3a admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:08.606464 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ca685-6892-4617-ab64-a51fedcb1d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:08.606739 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ca685-6892-4617-ab64-a51fedcb1d3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:08.607422 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ca685-6892-4617-ab64-a51fedcb1d3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:08.609014 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ca685-6892-4617-ab64-a51fedcb1d3a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:08.618801 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1082/2158] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:15:08.622635 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3034de-65da-40ad-adbf-80b82fab577d admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:08.632082 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3034de-65da-40ad-adbf-80b82fab577d admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:08.636945 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf50cf-9504-4b23-85a8-31a2ee1dd2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:08.637523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf50cf-9504-4b23-85a8-31a2ee1dd2a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:08.638305 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf50cf-9504-4b23-85a8-31a2ee1dd2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:08.638572 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf50cf-9504-4b23-85a8-31a2ee1dd2a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:08.639239 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf50cf-9504-4b23-85a8-31a2ee1dd2a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:08.640805 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf50cf-9504-4b23-85a8-31a2ee1dd2a8 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:08.651977 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1077/2159] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:15:08.657001 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ca685-6892-4617-ab64-a51fedcb1d3a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:08.667199 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ca685-6892-4617-ab64-a51fedcb1d3a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:08.671828 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ad841-11d3-4065-a10e-c68f72d87e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:08.672474 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ad841-11d3-4065-a10e-c68f72d87e40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:08.672755 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ad841-11d3-4065-a10e-c68f72d87e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:08.673030 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ad841-11d3-4065-a10e-c68f72d87e40 admin admin] PATH_INFO: `/v3/projects/303f80ab338841e4b4e173942048c164/users/1afa81e3784046e39c25df37dc2e8c19/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:08.689084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ad841-11d3-4065-a10e-c68f72d87e40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=303f80ab338841e4b4e173942048c164, user_id=1afa81e3784046e39c25df37dc2e8c19, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:08.691775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ad841-11d3-4065-a10e-c68f72d87e40 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:08.707278 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1083/2160] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:15:08 2022] PUT /identity/v3/projects/303f80ab338841e4b4e173942048c164/users/1afa81e3784046e39c25df37dc2e8c19/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:15:08.720760 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f7c36-5bd2-48e6-85ea-7a4deed7d75a None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:08.721131 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f7c36-5bd2-48e6-85ea-7a4deed7d75a None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:08.721807 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f7c36-5bd2-48e6-85ea-7a4deed7d75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:08.749298 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-052f7c36-5bd2-48e6-85ea-7a4deed7d75a None None] MFA Rules not processed for user `1afa81e3784046e39c25df37dc2e8c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:15:08.750747 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-052f7c36-5bd2-48e6-85ea-7a4deed7d75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:08.829596 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1078/2161] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:15:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:15:08.834079 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ad841-11d3-4065-a10e-c68f72d87e40 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:08.857478 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:15:08.904751 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ad841-11d3-4065-a10e-c68f72d87e40 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:08.909439 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c549f9bc-6824-4531-9300-306759157060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:08.910050 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549f9bc-6824-4531-9300-306759157060 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:08.910332 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549f9bc-6824-4531-9300-306759157060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:08.910608 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549f9bc-6824-4531-9300-306759157060 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:08.911278 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c549f9bc-6824-4531-9300-306759157060 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:08.913192 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c549f9bc-6824-4531-9300-306759157060 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:08.940065 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1084/2162] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:15:08 2022] POST /identity/v3/projects => generated 389 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:15:08.944194 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052f7c36-5bd2-48e6-85ea-7a4deed7d75a None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:08.957570 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052f7c36-5bd2-48e6-85ea-7a4deed7d75a None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:08.962692 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:15:09.021611 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81e2e4-da8f-4c08-8303-2a49c6b0e691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:09.022255 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81e2e4-da8f-4c08-8303-2a49c6b0e691 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:09.022535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81e2e4-da8f-4c08-8303-2a49c6b0e691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:09.022806 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f81e2e4-da8f-4c08-8303-2a49c6b0e691 admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:09.023499 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f81e2e4-da8f-4c08-8303-2a49c6b0e691 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:09.025422 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f81e2e4-da8f-4c08-8303-2a49c6b0e691 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:09.051005 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1079/2163] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:15:08 2022] POST /identity/v3/users => generated 301 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:15:09.051975 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c549f9bc-6824-4531-9300-306759157060 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:09.062998 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c549f9bc-6824-4531-9300-306759157060 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:09.067630 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4660b7-12d6-41b4-ad8b-52a1ef618a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:09.068224 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4660b7-12d6-41b4-ad8b-52a1ef618a2e admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:09.068505 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4660b7-12d6-41b4-ad8b-52a1ef618a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:09.068776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4660b7-12d6-41b4-ad8b-52a1ef618a2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:09.069547 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4660b7-12d6-41b4-ad8b-52a1ef618a2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:09.071087 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4660b7-12d6-41b4-ad8b-52a1ef618a2e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:09.082491 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1085/2164] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:15:09.086692 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81e2e4-da8f-4c08-8303-2a49c6b0e691 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:09.097282 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f81e2e4-da8f-4c08-8303-2a49c6b0e691 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:09.102004 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5fae06-6202-46df-aeb9-0fc2ddf9c589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:09.102789 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5fae06-6202-46df-aeb9-0fc2ddf9c589 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:09.103092 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5fae06-6202-46df-aeb9-0fc2ddf9c589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:09.103370 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5fae06-6202-46df-aeb9-0fc2ddf9c589 admin admin] PATH_INFO: `/v3/projects/295f8fe9589443f9bc562c907e0e05ff/users/a84b54c362ed4da5b0503e12ece66a3c/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:09.118478 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5fae06-6202-46df-aeb9-0fc2ddf9c589 admin admin] RBAC: Authorizing `identity:create_grant(project_id=295f8fe9589443f9bc562c907e0e05ff, user_id=a84b54c362ed4da5b0503e12ece66a3c, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:09.121168 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5fae06-6202-46df-aeb9-0fc2ddf9c589 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:09.133672 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1080/2165] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:15:09 2022] PUT /identity/v3/projects/295f8fe9589443f9bc562c907e0e05ff/users/a84b54c362ed4da5b0503e12ece66a3c/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:15:09.138280 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4660b7-12d6-41b4-ad8b-52a1ef618a2e admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:09.148042 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4660b7-12d6-41b4-ad8b-52a1ef618a2e admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:09.204037 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26dc630-755f-406f-9376-ee23c30cbc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:09.204665 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26dc630-755f-406f-9376-ee23c30cbc38 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:09.204945 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26dc630-755f-406f-9376-ee23c30cbc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:09.205232 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26dc630-755f-406f-9376-ee23c30cbc38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:09.205851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26dc630-755f-406f-9376-ee23c30cbc38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:09.207483 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26dc630-755f-406f-9376-ee23c30cbc38 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:09.217729 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1086/2166] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:15:09.221993 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5fae06-6202-46df-aeb9-0fc2ddf9c589 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:09.233148 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5fae06-6202-46df-aeb9-0fc2ddf9c589 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:09.238071 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1fcbf0-dd2d-4705-87c2-a7cae4ff234f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:09.238753 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1fcbf0-dd2d-4705-87c2-a7cae4ff234f admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:09.239028 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1fcbf0-dd2d-4705-87c2-a7cae4ff234f admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:09.239329 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1fcbf0-dd2d-4705-87c2-a7cae4ff234f admin admin] PATH_INFO: `/v3/projects/295f8fe9589443f9bc562c907e0e05ff/users/a84b54c362ed4da5b0503e12ece66a3c/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:09.241273 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1fcbf0-dd2d-4705-87c2-a7cae4ff234f admin admin] RBAC: Authorizing `identity:create_grant(project_id=295f8fe9589443f9bc562c907e0e05ff, user_id=a84b54c362ed4da5b0503e12ece66a3c, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:09.243909 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1fcbf0-dd2d-4705-87c2-a7cae4ff234f admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:09.256023 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1081/2167] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:15:09 2022] PUT /identity/v3/projects/295f8fe9589443f9bc562c907e0e05ff/users/a84b54c362ed4da5b0503e12ece66a3c/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:15:09.266322 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb6107-f1a5-4035-8176-d8b7fb3b5f1b None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:09.266637 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb6107-f1a5-4035-8176-d8b7fb3b5f1b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:09.266909 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb6107-f1a5-4035-8176-d8b7fb3b5f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:09.295945 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-9adb6107-f1a5-4035-8176-d8b7fb3b5f1b None None] MFA Rules not processed for user `a84b54c362ed4da5b0503e12ece66a3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:15:09.297139 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9adb6107-f1a5-4035-8176-d8b7fb3b5f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:09.363043 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1087/2168] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:15:09 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:15:09.374162 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1fcbf0-dd2d-4705-87c2-a7cae4ff234f admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:09.375874 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-6a1fcbf0-dd2d-4705-87c2-a7cae4ff234f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:09.431304 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1fcbf0-dd2d-4705-87c2-a7cae4ff234f admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:09.435421 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319be153-326a-4ec8-a1e4-e3081fea21b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:09.436011 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319be153-326a-4ec8-a1e4-e3081fea21b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:09.436287 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319be153-326a-4ec8-a1e4-e3081fea21b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:09.436551 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319be153-326a-4ec8-a1e4-e3081fea21b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:09.445201 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319be153-326a-4ec8-a1e4-e3081fea21b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:09.446614 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319be153-326a-4ec8-a1e4-e3081fea21b3 service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:09.455919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1082/2169] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:15:09 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:15:09.587383 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb6107-f1a5-4035-8176-d8b7fb3b5f1b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:09.600015 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:15:09.648739 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb6107-f1a5-4035-8176-d8b7fb3b5f1b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:09.653131 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec2324-b9c7-4bab-bdcd-827de3866b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:09.654257 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec2324-b9c7-4bab-bdcd-827de3866b40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:09.654531 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec2324-b9c7-4bab-bdcd-827de3866b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:09.654799 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec2324-b9c7-4bab-bdcd-827de3866b40 admin admin] PATH_INFO: `/v3/users/1afa81e3784046e39c25df37dc2e8c19` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:09.656028 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ec2324-b9c7-4bab-bdcd-827de3866b40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1afa81e3784046e39c25df37dc2e8c19)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:09.658410 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ec2324-b9c7-4bab-bdcd-827de3866b40 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:09.703712 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f3ec2324-b9c7-4bab-bdcd-827de3866b40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1afa81e3784046e39c25df37dc2e8c19'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:09.710449 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f3ec2324-b9c7-4bab-bdcd-827de3866b40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1afa81e3784046e39c25df37dc2e8c19'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:09.711140 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f3ec2324-b9c7-4bab-bdcd-827de3866b40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1afa81e3784046e39c25df37dc2e8c19'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:09.718809 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f3ec2324-b9c7-4bab-bdcd-827de3866b40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1afa81e3784046e39c25df37dc2e8c19'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:09.730456 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f3ec2324-b9c7-4bab-bdcd-827de3866b40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1afa81e3784046e39c25df37dc2e8c19'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:09.736683 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1088/2170] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:15:09 2022] DELETE /identity/v3/users/1afa81e3784046e39c25df37dc2e8c19 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:15:09.741110 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc1cee-df6b-4809-a004-5d8bc28c8881 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:09.741396 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc1cee-df6b-4809-a004-5d8bc28c8881 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:09.741670 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc1cee-df6b-4809-a004-5d8bc28c8881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:09.760282 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-66fc1cee-df6b-4809-a004-5d8bc28c8881 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:15:09.761482 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-66fc1cee-df6b-4809-a004-5d8bc28c8881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:09.820190 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1083/2171] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:15:09.829977 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec2324-b9c7-4bab-bdcd-827de3866b40 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:09.831537 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f3ec2324-b9c7-4bab-bdcd-827de3866b40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:09.896361 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec2324-b9c7-4bab-bdcd-827de3866b40 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:09.900106 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8946ccc-7507-4f1b-9f37-6b856c4ad9c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:09.900851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8946ccc-7507-4f1b-9f37-6b856c4ad9c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:09.901117 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8946ccc-7507-4f1b-9f37-6b856c4ad9c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:09.901432 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8946ccc-7507-4f1b-9f37-6b856c4ad9c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:09.917062 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8946ccc-7507-4f1b-9f37-6b856c4ad9c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:09.918657 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8946ccc-7507-4f1b-9f37-6b856c4ad9c8 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:09.935524 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1089/2172] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:15:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:15:10.216904 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fc1cee-df6b-4809-a004-5d8bc28c8881 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:10.218386 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-66fc1cee-df6b-4809-a004-5d8bc28c8881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:10.228776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:15:10.233075 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fc1cee-df6b-4809-a004-5d8bc28c8881 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:10.240278 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b98cfe-5bf5-4090-8aec-0dfb1f482bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:10.241224 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b98cfe-5bf5-4090-8aec-0dfb1f482bf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:10.241897 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b98cfe-5bf5-4090-8aec-0dfb1f482bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:10.242354 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b98cfe-5bf5-4090-8aec-0dfb1f482bf0 admin admin] PATH_INFO: `/v3/projects/303f80ab338841e4b4e173942048c164` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:10.243702 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b98cfe-5bf5-4090-8aec-0dfb1f482bf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=303f80ab338841e4b4e173942048c164)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:10.245716 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b98cfe-5bf5-4090-8aec-0dfb1f482bf0 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:10.293993 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-34b98cfe-5bf5-4090-8aec-0dfb1f482bf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '303f80ab338841e4b4e173942048c164', 'request_id': 'req-34b98cfe-5bf5-4090-8aec-0dfb1f482bf0'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:10.299713 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-34b98cfe-5bf5-4090-8aec-0dfb1f482bf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '303f80ab338841e4b4e173942048c164', 'request_id': 'req-34b98cfe-5bf5-4090-8aec-0dfb1f482bf0'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:10.305662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-34b98cfe-5bf5-4090-8aec-0dfb1f482bf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '303f80ab338841e4b4e173942048c164', 'request_id': 'req-34b98cfe-5bf5-4090-8aec-0dfb1f482bf0'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:10.317591 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-34b98cfe-5bf5-4090-8aec-0dfb1f482bf0 admin admin] The token cache is being invalidate because project 303f80ab338841e4b4e173942048c164 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:15:10.317879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-34b98cfe-5bf5-4090-8aec-0dfb1f482bf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:10.318534 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-34b98cfe-5bf5-4090-8aec-0dfb1f482bf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:10.320043 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1084/2173] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:15:10 2022] DELETE /identity/v3/projects/303f80ab338841e4b4e173942048c164 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:15:10.324124 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8946ccc-7507-4f1b-9f37-6b856c4ad9c8 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:10.334163 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8946ccc-7507-4f1b-9f37-6b856c4ad9c8 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:10.362690 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:15:10.390349 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31082c4-2807-4972-9516-8ec324272679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:10.390967 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31082c4-2807-4972-9516-8ec324272679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:10.391273 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31082c4-2807-4972-9516-8ec324272679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:10.391546 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31082c4-2807-4972-9516-8ec324272679 admin admin] PATH_INFO: `/v3/users/a84b54c362ed4da5b0503e12ece66a3c` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:10.392654 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31082c4-2807-4972-9516-8ec324272679 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a84b54c362ed4da5b0503e12ece66a3c)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:10.394569 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31082c4-2807-4972-9516-8ec324272679 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:10.439298 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d31082c4-2807-4972-9516-8ec324272679 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a84b54c362ed4da5b0503e12ece66a3c'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:10.446060 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d31082c4-2807-4972-9516-8ec324272679 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a84b54c362ed4da5b0503e12ece66a3c'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:10.446714 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d31082c4-2807-4972-9516-8ec324272679 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a84b54c362ed4da5b0503e12ece66a3c'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:10.454697 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d31082c4-2807-4972-9516-8ec324272679 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a84b54c362ed4da5b0503e12ece66a3c'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:10.466348 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d31082c4-2807-4972-9516-8ec324272679 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a84b54c362ed4da5b0503e12ece66a3c'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:10.472115 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1090/2174] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:15:10 2022] DELETE /identity/v3/users/a84b54c362ed4da5b0503e12ece66a3c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:15:10.749757 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b98cfe-5bf5-4090-8aec-0dfb1f482bf0 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:10.751507 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-34b98cfe-5bf5-4090-8aec-0dfb1f482bf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:10.764740 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b98cfe-5bf5-4090-8aec-0dfb1f482bf0 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:10.821246 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a75ec4-aceb-4f6d-b7cd-50b67ad1fe4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:10.821791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a75ec4-aceb-4f6d-b7cd-50b67ad1fe4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:10.822071 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a75ec4-aceb-4f6d-b7cd-50b67ad1fe4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:10.822340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a75ec4-aceb-4f6d-b7cd-50b67ad1fe4b admin admin] PATH_INFO: `/v3/projects/295f8fe9589443f9bc562c907e0e05ff` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:10.823492 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a75ec4-aceb-4f6d-b7cd-50b67ad1fe4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=295f8fe9589443f9bc562c907e0e05ff)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:10.825390 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a75ec4-aceb-4f6d-b7cd-50b67ad1fe4b admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:10.869471 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-68a75ec4-aceb-4f6d-b7cd-50b67ad1fe4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '295f8fe9589443f9bc562c907e0e05ff', 'request_id': 'req-68a75ec4-aceb-4f6d-b7cd-50b67ad1fe4b'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:10.875484 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-68a75ec4-aceb-4f6d-b7cd-50b67ad1fe4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '295f8fe9589443f9bc562c907e0e05ff', 'request_id': 'req-68a75ec4-aceb-4f6d-b7cd-50b67ad1fe4b'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:10.881366 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-68a75ec4-aceb-4f6d-b7cd-50b67ad1fe4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '295f8fe9589443f9bc562c907e0e05ff', 'request_id': 'req-68a75ec4-aceb-4f6d-b7cd-50b67ad1fe4b'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:10.893812 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-68a75ec4-aceb-4f6d-b7cd-50b67ad1fe4b admin admin] The token cache is being invalidate because project 295f8fe9589443f9bc562c907e0e05ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:15:10.894111 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-68a75ec4-aceb-4f6d-b7cd-50b67ad1fe4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:10.894906 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-68a75ec4-aceb-4f6d-b7cd-50b67ad1fe4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:10.896411 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1085/2175] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:15:10 2022] DELETE /identity/v3/projects/295f8fe9589443f9bc562c907e0e05ff => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:15:10.902078 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84bcce-4716-4ac0-b6b0-e2f0441aab41 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:10.902549 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84bcce-4716-4ac0-b6b0-e2f0441aab41 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:10.902646 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84bcce-4716-4ac0-b6b0-e2f0441aab41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:10.922909 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-fb84bcce-4716-4ac0-b6b0-e2f0441aab41 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:15:10.924819 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fb84bcce-4716-4ac0-b6b0-e2f0441aab41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:10.985880 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1091/2176] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:15:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:15:10.993472 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13a033-e648-459a-bca4-4985c54d3747 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:10.993755 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13a033-e648-459a-bca4-4985c54d3747 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:10.994026 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13a033-e648-459a-bca4-4985c54d3747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:11.014359 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-ce13a033-e648-459a-bca4-4985c54d3747 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:15:11.015578 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-ce13a033-e648-459a-bca4-4985c54d3747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:11.023441 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1086/2177] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:15:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:15:11.027328 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84bcce-4716-4ac0-b6b0-e2f0441aab41 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:11.037328 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84bcce-4716-4ac0-b6b0-e2f0441aab41 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:11.041867 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b05ffd-40d5-41fa-855d-c19d9d9c0082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:11.042461 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b05ffd-40d5-41fa-855d-c19d9d9c0082 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:11.042742 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b05ffd-40d5-41fa-855d-c19d9d9c0082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:11.043014 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b05ffd-40d5-41fa-855d-c19d9d9c0082 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:11.043701 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b05ffd-40d5-41fa-855d-c19d9d9c0082 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:15:11.044598 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b05ffd-40d5-41fa-855d-c19d9d9c0082 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:11.046199 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b05ffd-40d5-41fa-855d-c19d9d9c0082 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:11.057958 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1092/2178] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:15:11.062229 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce13a033-e648-459a-bca4-4985c54d3747 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:11.071368 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce13a033-e648-459a-bca4-4985c54d3747 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:11.075976 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3fcb5-c048-4573-9c1c-3c626892b2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:11.076562 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3fcb5-c048-4573-9c1c-3c626892b2c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:11.076840 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3fcb5-c048-4573-9c1c-3c626892b2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:11.077106 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3fcb5-c048-4573-9c1c-3c626892b2c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:11.077734 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e3fcb5-c048-4573-9c1c-3c626892b2c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:11.079660 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e3fcb5-c048-4573-9c1c-3c626892b2c4 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:11.088773 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:15:11.109896 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1087/2179] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:15:11 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:15:11.114012 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b05ffd-40d5-41fa-855d-c19d9d9c0082 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:11.124031 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b05ffd-40d5-41fa-855d-c19d9d9c0082 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:11.161805 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:15:11.183394 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e32e21-b997-4fcb-94e7-80aefcaa7fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:11.184095 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e32e21-b997-4fcb-94e7-80aefcaa7fd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:11.184467 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e32e21-b997-4fcb-94e7-80aefcaa7fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:11.184802 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e32e21-b997-4fcb-94e7-80aefcaa7fd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:11.185504 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e32e21-b997-4fcb-94e7-80aefcaa7fd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:11.187515 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e32e21-b997-4fcb-94e7-80aefcaa7fd7 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:11.209771 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1093/2180] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:15:11 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:15:11.214029 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3fcb5-c048-4573-9c1c-3c626892b2c4 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:11.225121 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3fcb5-c048-4573-9c1c-3c626892b2c4 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:11.229893 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa51459-a62e-4d5d-82c9-2888c57a2304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:11.230496 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa51459-a62e-4d5d-82c9-2888c57a2304 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:11.230779 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa51459-a62e-4d5d-82c9-2888c57a2304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:11.231182 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa51459-a62e-4d5d-82c9-2888c57a2304 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:11.232411 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa51459-a62e-4d5d-82c9-2888c57a2304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:11.234047 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa51459-a62e-4d5d-82c9-2888c57a2304 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:11.244197 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1088/2181] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:15:11.248124 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e32e21-b997-4fcb-94e7-80aefcaa7fd7 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:11.259211 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e32e21-b997-4fcb-94e7-80aefcaa7fd7 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:11.264282 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc60bd-c06f-4cdd-911e-bcfdcbd10d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:11.264880 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc60bd-c06f-4cdd-911e-bcfdcbd10d4d admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:11.265156 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc60bd-c06f-4cdd-911e-bcfdcbd10d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:11.265423 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc60bd-c06f-4cdd-911e-bcfdcbd10d4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:11.266055 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bc60bd-c06f-4cdd-911e-bcfdcbd10d4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:11.267674 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bc60bd-c06f-4cdd-911e-bcfdcbd10d4d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:11.281564 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1094/2182] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:15:11.283045 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa51459-a62e-4d5d-82c9-2888c57a2304 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:11.292888 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa51459-a62e-4d5d-82c9-2888c57a2304 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:11.298046 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787bee7c-1fa7-4ba9-af21-3e5913d06611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:11.298690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787bee7c-1fa7-4ba9-af21-3e5913d06611 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:11.298968 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787bee7c-1fa7-4ba9-af21-3e5913d06611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:11.299284 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787bee7c-1fa7-4ba9-af21-3e5913d06611 admin admin] PATH_INFO: `/v3/projects/5344aa60c73e4ef0ac75b528737f44a7/users/e69456f7db1d40a6a4e331afbbe90151/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:11.314287 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787bee7c-1fa7-4ba9-af21-3e5913d06611 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5344aa60c73e4ef0ac75b528737f44a7, user_id=e69456f7db1d40a6a4e331afbbe90151, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:11.316970 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787bee7c-1fa7-4ba9-af21-3e5913d06611 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:11.329716 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1089/2183] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:15:11 2022] PUT /identity/v3/projects/5344aa60c73e4ef0ac75b528737f44a7/users/e69456f7db1d40a6a4e331afbbe90151/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:15:11.341731 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16b693-8d9e-4ff0-9eaa-626af3cfb123 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:11.342016 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16b693-8d9e-4ff0-9eaa-626af3cfb123 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:11.342285 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16b693-8d9e-4ff0-9eaa-626af3cfb123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:11.369914 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-5b16b693-8d9e-4ff0-9eaa-626af3cfb123 None None] MFA Rules not processed for user `e69456f7db1d40a6a4e331afbbe90151`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:15:11.371143 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5b16b693-8d9e-4ff0-9eaa-626af3cfb123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:11.432401 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1095/2184] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:15:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:15:11.436990 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787bee7c-1fa7-4ba9-af21-3e5913d06611 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:11.498877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787bee7c-1fa7-4ba9-af21-3e5913d06611 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:11.503573 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba083b-11fd-4f1d-a27b-7febcc65e08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:11.504177 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba083b-11fd-4f1d-a27b-7febcc65e08a admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:11.504454 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba083b-11fd-4f1d-a27b-7febcc65e08a admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:11.504725 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba083b-11fd-4f1d-a27b-7febcc65e08a admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:11.505351 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cba083b-11fd-4f1d-a27b-7febcc65e08a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:11.507274 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cba083b-11fd-4f1d-a27b-7febcc65e08a admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:11.533550 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1090/2185] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:15:11 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:15:11.537662 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b16b693-8d9e-4ff0-9eaa-626af3cfb123 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:11.547927 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b16b693-8d9e-4ff0-9eaa-626af3cfb123 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:11.601901 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2493a18-e3dd-4b67-b8aa-58af2251adbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:11.602418 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2493a18-e3dd-4b67-b8aa-58af2251adbd admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:11.602693 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2493a18-e3dd-4b67-b8aa-58af2251adbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:11.602960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2493a18-e3dd-4b67-b8aa-58af2251adbd admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:11.603619 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2493a18-e3dd-4b67-b8aa-58af2251adbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:11.605499 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2493a18-e3dd-4b67-b8aa-58af2251adbd admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:11.626177 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1096/2186] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:15:11 2022] POST /identity/v3/users => generated 293 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:15:11.630416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba083b-11fd-4f1d-a27b-7febcc65e08a admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:11.640925 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba083b-11fd-4f1d-a27b-7febcc65e08a admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:11.645424 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ba9e5-b2fc-469a-9fbc-1c3af92b59b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:11.646007 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ba9e5-b2fc-469a-9fbc-1c3af92b59b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:11.646282 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ba9e5-b2fc-469a-9fbc-1c3af92b59b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:11.646551 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ba9e5-b2fc-469a-9fbc-1c3af92b59b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:11.647223 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ba9e5-b2fc-469a-9fbc-1c3af92b59b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:11.648787 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ba9e5-b2fc-469a-9fbc-1c3af92b59b8 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:11.660549 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1091/2187] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:15:11.664452 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2493a18-e3dd-4b67-b8aa-58af2251adbd admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:11.674101 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2493a18-e3dd-4b67-b8aa-58af2251adbd admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:11.678908 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f5b020-b48d-435b-8bb5-3b4d17f4bd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:11.679516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5b020-b48d-435b-8bb5-3b4d17f4bd2e admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:11.679792 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5b020-b48d-435b-8bb5-3b4d17f4bd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:11.680058 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f5b020-b48d-435b-8bb5-3b4d17f4bd2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:11.680685 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f5b020-b48d-435b-8bb5-3b4d17f4bd2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:11.682256 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f5b020-b48d-435b-8bb5-3b4d17f4bd2e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:11.691966 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1097/2188] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:15:11.695925 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ba9e5-b2fc-469a-9fbc-1c3af92b59b8 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:11.706154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ba9e5-b2fc-469a-9fbc-1c3af92b59b8 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:11.710668 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57fe82f-bfd9-425c-95a9-8aa9c9ed2601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:11.711327 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fe82f-bfd9-425c-95a9-8aa9c9ed2601 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:11.711608 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fe82f-bfd9-425c-95a9-8aa9c9ed2601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:11.711877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fe82f-bfd9-425c-95a9-8aa9c9ed2601 admin admin] PATH_INFO: `/v3/projects/745de7af2e6a4b9ab66e15e322c9ba61/users/0916cdf11ef947ed95c8c67f5bb216f1/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:11.726645 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57fe82f-bfd9-425c-95a9-8aa9c9ed2601 admin admin] RBAC: Authorizing `identity:create_grant(project_id=745de7af2e6a4b9ab66e15e322c9ba61, user_id=0916cdf11ef947ed95c8c67f5bb216f1, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:11.729293 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57fe82f-bfd9-425c-95a9-8aa9c9ed2601 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:11.741572 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1092/2189] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:15:11 2022] PUT /identity/v3/projects/745de7af2e6a4b9ab66e15e322c9ba61/users/0916cdf11ef947ed95c8c67f5bb216f1/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:15:11.752138 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cc3e1-8708-490c-870d-35176e1b159c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:11.752424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cc3e1-8708-490c-870d-35176e1b159c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:11.752698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cc3e1-8708-490c-870d-35176e1b159c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:11.780035 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-7c7cc3e1-8708-490c-870d-35176e1b159c None None] MFA Rules not processed for user `0916cdf11ef947ed95c8c67f5bb216f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:15:11.781223 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7c7cc3e1-8708-490c-870d-35176e1b159c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:11.793260 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:15:11.844383 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1098/2190] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:15:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:15:11.848940 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57fe82f-bfd9-425c-95a9-8aa9c9ed2601 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:11.909177 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57fe82f-bfd9-425c-95a9-8aa9c9ed2601 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:11.913843 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5b979-ea5f-4582-8234-c63049bfdf67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:11.914441 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5b979-ea5f-4582-8234-c63049bfdf67 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:11.914719 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5b979-ea5f-4582-8234-c63049bfdf67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:11.914988 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5b979-ea5f-4582-8234-c63049bfdf67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:11.915664 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5b979-ea5f-4582-8234-c63049bfdf67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:11.917550 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5b979-ea5f-4582-8234-c63049bfdf67 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:11.928781 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:15:11.945683 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1093/2191] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:15:11 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:15:11.949660 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cc3e1-8708-490c-870d-35176e1b159c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:11.961833 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cc3e1-8708-490c-870d-35176e1b159c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:12.019117 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc81003-6106-4246-a21d-c503083bb617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:12.019727 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc81003-6106-4246-a21d-c503083bb617 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:12.020007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc81003-6106-4246-a21d-c503083bb617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:12.020279 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc81003-6106-4246-a21d-c503083bb617 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:12.020916 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc81003-6106-4246-a21d-c503083bb617 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:12.022865 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc81003-6106-4246-a21d-c503083bb617 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:12.043407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1099/2192] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:15:11 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:15:12.047405 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5b979-ea5f-4582-8234-c63049bfdf67 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:12.058129 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5b979-ea5f-4582-8234-c63049bfdf67 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:12.063286 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc82b2a-ed7a-4afc-aa26-285c69b152ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:12.063879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc82b2a-ed7a-4afc-aa26-285c69b152ab admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:12.064162 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc82b2a-ed7a-4afc-aa26-285c69b152ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:12.064435 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc82b2a-ed7a-4afc-aa26-285c69b152ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:12.065075 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc82b2a-ed7a-4afc-aa26-285c69b152ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:12.066662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc82b2a-ed7a-4afc-aa26-285c69b152ab admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:12.077420 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1094/2193] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:15:12.081510 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc81003-6106-4246-a21d-c503083bb617 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:12.091373 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc81003-6106-4246-a21d-c503083bb617 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:12.096004 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e5ed6-eafe-4774-acdd-8717fdd68c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:12.096641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e5ed6-eafe-4774-acdd-8717fdd68c27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:12.096922 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e5ed6-eafe-4774-acdd-8717fdd68c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:12.097196 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e5ed6-eafe-4774-acdd-8717fdd68c27 admin admin] PATH_INFO: `/v3/projects/1eeb3a4794b64ed897bc7abb561ef701/users/f32a124f86ec47e28c92cbabecaab313/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:12.111941 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e5ed6-eafe-4774-acdd-8717fdd68c27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eeb3a4794b64ed897bc7abb561ef701, user_id=f32a124f86ec47e28c92cbabecaab313, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:12.114565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e5ed6-eafe-4774-acdd-8717fdd68c27 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:12.127939 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1100/2194] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:15:12 2022] PUT /identity/v3/projects/1eeb3a4794b64ed897bc7abb561ef701/users/f32a124f86ec47e28c92cbabecaab313/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:15:12.131799 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc82b2a-ed7a-4afc-aa26-285c69b152ab admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:12.141731 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc82b2a-ed7a-4afc-aa26-285c69b152ab admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:12.197607 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a44bacc-869f-4612-950d-23f17ce35492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:12.198132 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44bacc-869f-4612-950d-23f17ce35492 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:12.198410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44bacc-869f-4612-950d-23f17ce35492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:12.198675 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44bacc-869f-4612-950d-23f17ce35492 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:12.199360 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a44bacc-869f-4612-950d-23f17ce35492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:12.200936 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a44bacc-869f-4612-950d-23f17ce35492 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:12.210770 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1095/2195] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:15:12.214808 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e5ed6-eafe-4774-acdd-8717fdd68c27 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:12.224931 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e5ed6-eafe-4774-acdd-8717fdd68c27 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:12.229593 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d4295-872d-4fb3-8b1f-ed36a4bcc4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:12.230231 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d4295-872d-4fb3-8b1f-ed36a4bcc4c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:12.230537 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d4295-872d-4fb3-8b1f-ed36a4bcc4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:12.230804 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d4295-872d-4fb3-8b1f-ed36a4bcc4c1 admin admin] PATH_INFO: `/v3/projects/1eeb3a4794b64ed897bc7abb561ef701/users/f32a124f86ec47e28c92cbabecaab313/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:12.233244 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d4295-872d-4fb3-8b1f-ed36a4bcc4c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eeb3a4794b64ed897bc7abb561ef701, user_id=f32a124f86ec47e28c92cbabecaab313, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:12.235940 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d4295-872d-4fb3-8b1f-ed36a4bcc4c1 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:12.245958 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1101/2196] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:15:12 2022] PUT /identity/v3/projects/1eeb3a4794b64ed897bc7abb561ef701/users/f32a124f86ec47e28c92cbabecaab313/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:15:12.257586 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b6bb3-1676-456b-a559-19733d9ac8c1 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:12.258079 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b6bb3-1676-456b-a559-19733d9ac8c1 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:12.258173 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b6bb3-1676-456b-a559-19733d9ac8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:12.285597 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-b66b6bb3-1676-456b-a559-19733d9ac8c1 None None] MFA Rules not processed for user `f32a124f86ec47e28c92cbabecaab313`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:15:12.286912 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-b66b6bb3-1676-456b-a559-19733d9ac8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:12.355655 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1096/2197] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:15:12 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:15:12.367932 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d4295-872d-4fb3-8b1f-ed36a4bcc4c1 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:12.369605 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-726d4295-872d-4fb3-8b1f-ed36a4bcc4c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:12.426627 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d4295-872d-4fb3-8b1f-ed36a4bcc4c1 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:12.430376 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33695bec-31b4-48ed-ad90-7022ada56804 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:12.430972 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33695bec-31b4-48ed-ad90-7022ada56804 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:12.431276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33695bec-31b4-48ed-ad90-7022ada56804 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:12.431544 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33695bec-31b4-48ed-ad90-7022ada56804 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:12.452127 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:15:12.488481 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33695bec-31b4-48ed-ad90-7022ada56804 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:12.489900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33695bec-31b4-48ed-ad90-7022ada56804 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:12.498580 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1102/2198] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:15:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:15:13.368474 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b6bb3-1676-456b-a559-19733d9ac8c1 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:13.370028 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-b66b6bb3-1676-456b-a559-19733d9ac8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:13.425440 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b6bb3-1676-456b-a559-19733d9ac8c1 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:13.428927 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee886a-e8f0-4699-a674-d4b9e0336eab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:13.429524 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee886a-e8f0-4699-a674-d4b9e0336eab service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:13.429799 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee886a-e8f0-4699-a674-d4b9e0336eab service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:13.430068 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee886a-e8f0-4699-a674-d4b9e0336eab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:13.431786 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-8eee886a-e8f0-4699-a674-d4b9e0336eab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:13.441324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee886a-e8f0-4699-a674-d4b9e0336eab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:13.442727 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee886a-e8f0-4699-a674-d4b9e0336eab service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:13.450381 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1097/2199] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:15:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:15:13.980901 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33695bec-31b4-48ed-ad90-7022ada56804 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:13.980901 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-33695bec-31b4-48ed-ad90-7022ada56804 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:14.011778 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee886a-e8f0-4699-a674-d4b9e0336eab service neutron] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:14.013369 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-8eee886a-e8f0-4699-a674-d4b9e0336eab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:14.031430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:15:14.038651 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee886a-e8f0-4699-a674-d4b9e0336eab service neutron] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:14.042909 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2634cc9-0106-4cef-bdfa-afc483d21726 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:14.043551 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2634cc9-0106-4cef-bdfa-afc483d21726 service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:14.043818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2634cc9-0106-4cef-bdfa-afc483d21726 service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:14.044087 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2634cc9-0106-4cef-bdfa-afc483d21726 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:14.071236 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33695bec-31b4-48ed-ad90-7022ada56804 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:14.076405 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a81f9-98f8-4207-9902-b945cee4b274 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a971cdb7f9d4e6692c6326c103d7303', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:14.077514 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a81f9-98f8-4207-9902-b945cee4b274 service placement] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:14.077805 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a81f9-98f8-4207-9902-b945cee4b274 service placement] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:14.078081 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a81f9-98f8-4207-9902-b945cee4b274 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:14.079996 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4c6a81f9-98f8-4207-9902-b945cee4b274 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:14.091291 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a81f9-98f8-4207-9902-b945cee4b274 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:14.093336 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a81f9-98f8-4207-9902-b945cee4b274 service placement] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:14.101565 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2634cc9-0106-4cef-bdfa-afc483d21726 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:14.102997 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2634cc9-0106-4cef-bdfa-afc483d21726 service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:14.103129 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1103/2200] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:15:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:15:14.112574 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1098/2201] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:15:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 13 13:15:14.653218 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a81f9-98f8-4207-9902-b945cee4b274 service placement] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:14.664012 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a81f9-98f8-4207-9902-b945cee4b274 service placement] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:14.667934 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337252be-9499-4263-b216-811c91d0f7e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:14.668852 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337252be-9499-4263-b216-811c91d0f7e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:14.669109 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337252be-9499-4263-b216-811c91d0f7e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:14.669385 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337252be-9499-4263-b216-811c91d0f7e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:14.680932 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337252be-9499-4263-b216-811c91d0f7e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:14.682421 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337252be-9499-4263-b216-811c91d0f7e1 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:14.694034 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1104/2202] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:15:14 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:15:15.864276 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7e509-4177-4c8d-96ed-dc82c7ae3e17 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:15.865198 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7e509-4177-4c8d-96ed-dc82c7ae3e17 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:15.865700 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac7e509-4177-4c8d-96ed-dc82c7ae3e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:15.889905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-dac7e509-4177-4c8d-96ed-dc82c7ae3e17 None None] MFA Rules not processed for user `b6ae674f0efc445fa970fca64e13893a`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:15:15.892227 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-dac7e509-4177-4c8d-96ed-dc82c7ae3e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:15.974778 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1099/2203] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Oct 13 13:15:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:15:15.986596 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337252be-9499-4263-b216-811c91d0f7e1 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:15.993890 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-337252be-9499-4263-b216-811c91d0f7e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:16.119965 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337252be-9499-4263-b216-811c91d0f7e1 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:16.126278 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5385336-a9eb-4983-8dc2-1707f9f0f1d0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526000341fd146db8c661f1fd14d4801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:16.127027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5385336-a9eb-4983-8dc2-1707f9f0f1d0 service swift] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:16.127431 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5385336-a9eb-4983-8dc2-1707f9f0f1d0 service swift] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:16.129093 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5385336-a9eb-4983-8dc2-1707f9f0f1d0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:16.130747 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac7e509-4177-4c8d-96ed-dc82c7ae3e17 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:16.133840 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-dac7e509-4177-4c8d-96ed-dc82c7ae3e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:16.166401 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5385336-a9eb-4983-8dc2-1707f9f0f1d0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:16.166894 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac7e509-4177-4c8d-96ed-dc82c7ae3e17 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:16.168228 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5385336-a9eb-4983-8dc2-1707f9f0f1d0 service swift] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:16.189309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1105/2204] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 13 13:15:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 13 13:15:16.194021 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c927b-9060-4316-bcc6-625586428b2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:16.194863 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c927b-9060-4316-bcc6-625586428b2f service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:16.195315 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c927b-9060-4316-bcc6-625586428b2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:16.195744 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c927b-9060-4316-bcc6-625586428b2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:16.204390 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-ca6c927b-9060-4316-bcc6-625586428b2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:16.219565 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c927b-9060-4316-bcc6-625586428b2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:16.221498 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c927b-9060-4316-bcc6-625586428b2f service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:16.236721 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1100/2205] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:15:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:15:19.034586 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5385336-a9eb-4983-8dc2-1707f9f0f1d0 service swift] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:19.069395 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:15:19.104782 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5385336-a9eb-4983-8dc2-1707f9f0f1d0 service swift] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:19.109741 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e6c826-13b6-4310-ae44-06f5d744fcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:19.110401 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6c826-13b6-4310-ae44-06f5d744fcb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:19.110680 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6c826-13b6-4310-ae44-06f5d744fcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:19.110952 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6c826-13b6-4310-ae44-06f5d744fcb1 admin admin] PATH_INFO: `/v3/users/e69456f7db1d40a6a4e331afbbe90151` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:19.112175 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e6c826-13b6-4310-ae44-06f5d744fcb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e69456f7db1d40a6a4e331afbbe90151)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:19.114120 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e6c826-13b6-4310-ae44-06f5d744fcb1 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:19.171089 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-56e6c826-13b6-4310-ae44-06f5d744fcb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e69456f7db1d40a6a4e331afbbe90151'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:19.178812 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-56e6c826-13b6-4310-ae44-06f5d744fcb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e69456f7db1d40a6a4e331afbbe90151'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:19.179578 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-56e6c826-13b6-4310-ae44-06f5d744fcb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e69456f7db1d40a6a4e331afbbe90151'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:19.188077 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-56e6c826-13b6-4310-ae44-06f5d744fcb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e69456f7db1d40a6a4e331afbbe90151'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:19.203749 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-56e6c826-13b6-4310-ae44-06f5d744fcb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e69456f7db1d40a6a4e331afbbe90151'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:19.210979 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1106/2206] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:15:19 2022] DELETE /identity/v3/users/e69456f7db1d40a6a4e331afbbe90151 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:15:19.216928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdcc270-2a04-4f82-bff5-edfc4e6319e4 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:19.217410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdcc270-2a04-4f82-bff5-edfc4e6319e4 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:19.217507 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdcc270-2a04-4f82-bff5-edfc4e6319e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:19.241613 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-6fdcc270-2a04-4f82-bff5-edfc4e6319e4 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:15:19.242890 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-6fdcc270-2a04-4f82-bff5-edfc4e6319e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:19.319146 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1101/2207] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:15:19.322993 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e6c826-13b6-4310-ae44-06f5d744fcb1 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:19.324476 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-56e6c826-13b6-4310-ae44-06f5d744fcb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:19.403703 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e6c826-13b6-4310-ae44-06f5d744fcb1 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:19.407323 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdac98d-f2ae-4381-9c33-68731edc248f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:19.407925 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdac98d-f2ae-4381-9c33-68731edc248f service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:19.408203 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdac98d-f2ae-4381-9c33-68731edc248f service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:19.408477 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdac98d-f2ae-4381-9c33-68731edc248f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:19.417779 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdac98d-f2ae-4381-9c33-68731edc248f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:19.419366 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdac98d-f2ae-4381-9c33-68731edc248f service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:19.429753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1107/2208] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:15:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:15:19.992236 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdcc270-2a04-4f82-bff5-edfc4e6319e4 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:19.994196 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-6fdcc270-2a04-4f82-bff5-edfc4e6319e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:20.008204 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdcc270-2a04-4f82-bff5-edfc4e6319e4 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:20.013284 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3453015e-70ef-42b9-8720-c62b128ebbd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:20.014067 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453015e-70ef-42b9-8720-c62b128ebbd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:20.014451 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453015e-70ef-42b9-8720-c62b128ebbd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:20.014920 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453015e-70ef-42b9-8720-c62b128ebbd1 admin admin] PATH_INFO: `/v3/projects/5344aa60c73e4ef0ac75b528737f44a7` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:20.018106 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3453015e-70ef-42b9-8720-c62b128ebbd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5344aa60c73e4ef0ac75b528737f44a7)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:20.019904 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3453015e-70ef-42b9-8720-c62b128ebbd1 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:20.077264 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:15:20.099100 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-3453015e-70ef-42b9-8720-c62b128ebbd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5344aa60c73e4ef0ac75b528737f44a7', 'request_id': 'req-3453015e-70ef-42b9-8720-c62b128ebbd1'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:20.111402 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-3453015e-70ef-42b9-8720-c62b128ebbd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5344aa60c73e4ef0ac75b528737f44a7', 'request_id': 'req-3453015e-70ef-42b9-8720-c62b128ebbd1'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:20.118514 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-3453015e-70ef-42b9-8720-c62b128ebbd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5344aa60c73e4ef0ac75b528737f44a7', 'request_id': 'req-3453015e-70ef-42b9-8720-c62b128ebbd1'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:20.140066 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-3453015e-70ef-42b9-8720-c62b128ebbd1 admin admin] The token cache is being invalidate because project 5344aa60c73e4ef0ac75b528737f44a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:15:20.140480 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-3453015e-70ef-42b9-8720-c62b128ebbd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:20.141200 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-3453015e-70ef-42b9-8720-c62b128ebbd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:20.143174 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1102/2209] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:15:19 2022] DELETE /identity/v3/projects/5344aa60c73e4ef0ac75b528737f44a7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:15:20.147127 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdac98d-f2ae-4381-9c33-68731edc248f service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:20.159510 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdac98d-f2ae-4381-9c33-68731edc248f service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:20.226637 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:15:20.240051 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c2a8e9-f428-4bd3-b58c-75eab7a9af33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:20.241092 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2a8e9-f428-4bd3-b58c-75eab7a9af33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:20.241713 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2a8e9-f428-4bd3-b58c-75eab7a9af33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:20.242352 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2a8e9-f428-4bd3-b58c-75eab7a9af33 admin admin] PATH_INFO: `/v3/users/0916cdf11ef947ed95c8c67f5bb216f1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:20.243789 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c2a8e9-f428-4bd3-b58c-75eab7a9af33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0916cdf11ef947ed95c8c67f5bb216f1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:20.245982 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c2a8e9-f428-4bd3-b58c-75eab7a9af33 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:20.298878 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-91c2a8e9-f428-4bd3-b58c-75eab7a9af33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0916cdf11ef947ed95c8c67f5bb216f1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:20.306408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-91c2a8e9-f428-4bd3-b58c-75eab7a9af33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0916cdf11ef947ed95c8c67f5bb216f1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:20.307139 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-91c2a8e9-f428-4bd3-b58c-75eab7a9af33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0916cdf11ef947ed95c8c67f5bb216f1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:20.314993 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-91c2a8e9-f428-4bd3-b58c-75eab7a9af33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0916cdf11ef947ed95c8c67f5bb216f1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:20.328562 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-91c2a8e9-f428-4bd3-b58c-75eab7a9af33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0916cdf11ef947ed95c8c67f5bb216f1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:20.334728 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1108/2210] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:15:20 2022] DELETE /identity/v3/users/0916cdf11ef947ed95c8c67f5bb216f1 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:15:20.499865 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3453015e-70ef-42b9-8720-c62b128ebbd1 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:20.501611 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-3453015e-70ef-42b9-8720-c62b128ebbd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:20.603481 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3453015e-70ef-42b9-8720-c62b128ebbd1 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:20.634677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c2a8e9-f428-4bd3-b58c-75eab7a9af33 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:20.636279 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-91c2a8e9-f428-4bd3-b58c-75eab7a9af33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:20.721394 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc09b82-2e88-4283-ac9a-b621580f6804 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:20.722215 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc09b82-2e88-4283-ac9a-b621580f6804 service nova] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:20.722572 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc09b82-2e88-4283-ac9a-b621580f6804 service nova] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:20.723107 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc09b82-2e88-4283-ac9a-b621580f6804 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:20.725255 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-bbc09b82-2e88-4283-ac9a-b621580f6804 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:20.736666 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c2a8e9-f428-4bd3-b58c-75eab7a9af33 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:20.744709 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e63cb-7d49-4469-b283-da435dbe6ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:20.745282 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e63cb-7d49-4469-b283-da435dbe6ba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:20.745564 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e63cb-7d49-4469-b283-da435dbe6ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:20.745840 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e63cb-7d49-4469-b283-da435dbe6ba7 admin admin] PATH_INFO: `/v3/projects/745de7af2e6a4b9ab66e15e322c9ba61` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:20.746997 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e63cb-7d49-4469-b283-da435dbe6ba7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=745de7af2e6a4b9ab66e15e322c9ba61)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:20.749025 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e63cb-7d49-4469-b283-da435dbe6ba7 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:20.789773 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:15:20.804303 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc09b82-2e88-4283-ac9a-b621580f6804 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:20.806764 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc09b82-2e88-4283-ac9a-b621580f6804 service nova] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:20.814165 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-523e63cb-7d49-4469-b283-da435dbe6ba7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '745de7af2e6a4b9ab66e15e322c9ba61', 'request_id': 'req-523e63cb-7d49-4469-b283-da435dbe6ba7'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:20.817380 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1103/2211] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:15:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:15:20.829906 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-523e63cb-7d49-4469-b283-da435dbe6ba7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '745de7af2e6a4b9ab66e15e322c9ba61', 'request_id': 'req-523e63cb-7d49-4469-b283-da435dbe6ba7'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:20.837454 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-523e63cb-7d49-4469-b283-da435dbe6ba7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '745de7af2e6a4b9ab66e15e322c9ba61', 'request_id': 'req-523e63cb-7d49-4469-b283-da435dbe6ba7'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:20.850675 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:15:20.856700 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-523e63cb-7d49-4469-b283-da435dbe6ba7 admin admin] The token cache is being invalidate because project 745de7af2e6a4b9ab66e15e322c9ba61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:15:20.858201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-523e63cb-7d49-4469-b283-da435dbe6ba7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:20.859473 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-523e63cb-7d49-4469-b283-da435dbe6ba7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:20.861498 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1109/2212] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:15:20 2022] DELETE /identity/v3/projects/745de7af2e6a4b9ab66e15e322c9ba61 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:15:20.865385 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc09b82-2e88-4283-ac9a-b621580f6804 service nova] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:20.877542 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc09b82-2e88-4283-ac9a-b621580f6804 service nova] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:20.961365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfefbc0-fad8-4dc0-adfa-b7bc773ef9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:20.962188 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfefbc0-fad8-4dc0-adfa-b7bc773ef9fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:20.962569 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfefbc0-fad8-4dc0-adfa-b7bc773ef9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:20.962941 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfefbc0-fad8-4dc0-adfa-b7bc773ef9fe admin admin] PATH_INFO: `/v3/users/f32a124f86ec47e28c92cbabecaab313` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:20.964233 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfefbc0-fad8-4dc0-adfa-b7bc773ef9fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=f32a124f86ec47e28c92cbabecaab313)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:20.966362 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfefbc0-fad8-4dc0-adfa-b7bc773ef9fe admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:21.055799 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-edfefbc0-fad8-4dc0-adfa-b7bc773ef9fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f32a124f86ec47e28c92cbabecaab313'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:21.065713 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-edfefbc0-fad8-4dc0-adfa-b7bc773ef9fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f32a124f86ec47e28c92cbabecaab313'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:21.066714 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-edfefbc0-fad8-4dc0-adfa-b7bc773ef9fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f32a124f86ec47e28c92cbabecaab313'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:21.067924 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-edfefbc0-fad8-4dc0-adfa-b7bc773ef9fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f32a124f86ec47e28c92cbabecaab313'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:21.078955 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-edfefbc0-fad8-4dc0-adfa-b7bc773ef9fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f32a124f86ec47e28c92cbabecaab313'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:21.114141 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1104/2213] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:15:20 2022] DELETE /identity/v3/users/f32a124f86ec47e28c92cbabecaab313 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:15:21.519887 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e63cb-7d49-4469-b283-da435dbe6ba7 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:21.521357 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-523e63cb-7d49-4469-b283-da435dbe6ba7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:21.538744 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e63cb-7d49-4469-b283-da435dbe6ba7 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:21.612764 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3477ebae-4d0b-4c6a-8c32-136bba610de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:21.613389 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477ebae-4d0b-4c6a-8c32-136bba610de6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:21.613672 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477ebae-4d0b-4c6a-8c32-136bba610de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:21.614532 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477ebae-4d0b-4c6a-8c32-136bba610de6 admin admin] PATH_INFO: `/v3/projects/1eeb3a4794b64ed897bc7abb561ef701` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:21.615702 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3477ebae-4d0b-4c6a-8c32-136bba610de6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1eeb3a4794b64ed897bc7abb561ef701)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:21.617649 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3477ebae-4d0b-4c6a-8c32-136bba610de6 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:21.672427 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3477ebae-4d0b-4c6a-8c32-136bba610de6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1eeb3a4794b64ed897bc7abb561ef701', 'request_id': 'req-3477ebae-4d0b-4c6a-8c32-136bba610de6'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:21.679630 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3477ebae-4d0b-4c6a-8c32-136bba610de6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1eeb3a4794b64ed897bc7abb561ef701', 'request_id': 'req-3477ebae-4d0b-4c6a-8c32-136bba610de6'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:21.686385 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3477ebae-4d0b-4c6a-8c32-136bba610de6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1eeb3a4794b64ed897bc7abb561ef701', 'request_id': 'req-3477ebae-4d0b-4c6a-8c32-136bba610de6'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:21.702108 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3477ebae-4d0b-4c6a-8c32-136bba610de6 admin admin] The token cache is being invalidate because project 1eeb3a4794b64ed897bc7abb561ef701 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:15:21.703338 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3477ebae-4d0b-4c6a-8c32-136bba610de6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:21.704178 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3477ebae-4d0b-4c6a-8c32-136bba610de6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:15:21.705979 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1110/2214] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:15:21 2022] DELETE /identity/v3/projects/1eeb3a4794b64ed897bc7abb561ef701 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:15:21.711032 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c2d66-103c-4beb-8dcb-329795842926 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:21.711565 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c2d66-103c-4beb-8dcb-329795842926 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:21.711877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c2d66-103c-4beb-8dcb-329795842926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:21.733508 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-b01c2d66-103c-4beb-8dcb-329795842926 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:15:21.734831 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-b01c2d66-103c-4beb-8dcb-329795842926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:21.805116 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1105/2215] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:15:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:15:21.812864 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140666d5-7a8a-443a-9be0-386bbc6b096c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:21.815535 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140666d5-7a8a-443a-9be0-386bbc6b096c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:21.815974 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140666d5-7a8a-443a-9be0-386bbc6b096c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:21.842971 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-140666d5-7a8a-443a-9be0-386bbc6b096c None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:15:21.844388 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-140666d5-7a8a-443a-9be0-386bbc6b096c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:21.858134 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1111/2216] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:15:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:15:21.859206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c2d66-103c-4beb-8dcb-329795842926 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:21.869481 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c2d66-103c-4beb-8dcb-329795842926 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:21.874641 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d68953-b125-413f-8ee1-df36ed3c0dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:21.875374 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d68953-b125-413f-8ee1-df36ed3c0dab admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:21.875746 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d68953-b125-413f-8ee1-df36ed3c0dab admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:21.876092 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d68953-b125-413f-8ee1-df36ed3c0dab admin admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:21.876816 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d68953-b125-413f-8ee1-df36ed3c0dab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:15:21.877299 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d68953-b125-413f-8ee1-df36ed3c0dab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:21.878976 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d68953-b125-413f-8ee1-df36ed3c0dab admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:21.886991 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:15:21.895931 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1106/2217] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:15:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 13:15:21.898121 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140666d5-7a8a-443a-9be0-386bbc6b096c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:21.909464 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140666d5-7a8a-443a-9be0-386bbc6b096c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:21.915279 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817797d-04c5-46cc-b29d-6ba7b371467b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:21.916007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817797d-04c5-46cc-b29d-6ba7b371467b admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:21.916389 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817797d-04c5-46cc-b29d-6ba7b371467b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:21.916790 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817797d-04c5-46cc-b29d-6ba7b371467b admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:21.917510 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817797d-04c5-46cc-b29d-6ba7b371467b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:21.919740 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817797d-04c5-46cc-b29d-6ba7b371467b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:21.958433 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1112/2218] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:15:21 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:15:21.960211 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d68953-b125-413f-8ee1-df36ed3c0dab admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:21.971555 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d68953-b125-413f-8ee1-df36ed3c0dab admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:22.042794 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2081ba5b-6b62-4b98-962a-12d077bea026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:22.043711 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081ba5b-6b62-4b98-962a-12d077bea026 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:22.043990 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081ba5b-6b62-4b98-962a-12d077bea026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:22.044265 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081ba5b-6b62-4b98-962a-12d077bea026 admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:22.045014 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2081ba5b-6b62-4b98-962a-12d077bea026 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:22.047001 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2081ba5b-6b62-4b98-962a-12d077bea026 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:22.080536 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1107/2219] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:15:21 2022] POST /identity/v3/users => generated 286 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:15:22.081653 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817797d-04c5-46cc-b29d-6ba7b371467b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:22.098922 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817797d-04c5-46cc-b29d-6ba7b371467b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:22.103641 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c914e7-5f6a-4f14-a7c3-794e64f080e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:22.104241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c914e7-5f6a-4f14-a7c3-794e64f080e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:22.104562 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c914e7-5f6a-4f14-a7c3-794e64f080e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:22.104810 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c914e7-5f6a-4f14-a7c3-794e64f080e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:22.105447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c914e7-5f6a-4f14-a7c3-794e64f080e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:22.107048 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c914e7-5f6a-4f14-a7c3-794e64f080e1 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:22.121064 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1113/2220] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:15:22.124283 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2081ba5b-6b62-4b98-962a-12d077bea026 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:22.136456 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2081ba5b-6b62-4b98-962a-12d077bea026 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:22.142608 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e00ec1-35b1-4f35-9fa4-5d6f6905304d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:22.143386 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e00ec1-35b1-4f35-9fa4-5d6f6905304d admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:22.143750 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e00ec1-35b1-4f35-9fa4-5d6f6905304d admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:22.144112 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e00ec1-35b1-4f35-9fa4-5d6f6905304d admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:22.144937 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e00ec1-35b1-4f35-9fa4-5d6f6905304d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:22.146626 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e00ec1-35b1-4f35-9fa4-5d6f6905304d admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:22.164821 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1108/2221] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:15:22.166070 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c914e7-5f6a-4f14-a7c3-794e64f080e1 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:22.178934 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c914e7-5f6a-4f14-a7c3-794e64f080e1 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:22.183649 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef2207-0fe2-4e82-b0e8-60388620aaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:22.184300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef2207-0fe2-4e82-b0e8-60388620aaec admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:22.184762 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef2207-0fe2-4e82-b0e8-60388620aaec admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:22.184963 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef2207-0fe2-4e82-b0e8-60388620aaec admin admin] PATH_INFO: `/v3/projects/12163a99662b48759b3fd7da51445ecf/users/18920ad97e6946df9689bd6d4abd50b0/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:22.203832 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef2207-0fe2-4e82-b0e8-60388620aaec admin admin] RBAC: Authorizing `identity:create_grant(project_id=12163a99662b48759b3fd7da51445ecf, user_id=18920ad97e6946df9689bd6d4abd50b0, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:22.206912 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef2207-0fe2-4e82-b0e8-60388620aaec admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:22.231029 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1114/2222] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:15:22 2022] PUT /identity/v3/projects/12163a99662b48759b3fd7da51445ecf/users/18920ad97e6946df9689bd6d4abd50b0/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:15:22.235941 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55571b24-1042-4285-84a4-8dbb207b5b79 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:22.236182 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55571b24-1042-4285-84a4-8dbb207b5b79 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:22.236458 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55571b24-1042-4285-84a4-8dbb207b5b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:22.258712 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-55571b24-1042-4285-84a4-8dbb207b5b79 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:15:22.260015 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-55571b24-1042-4285-84a4-8dbb207b5b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:22.350029 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1109/2223] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:15:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:15:22.361137 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef2207-0fe2-4e82-b0e8-60388620aaec admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:22.364249 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-13ef2207-0fe2-4e82-b0e8-60388620aaec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:22.399281 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:15:22.445908 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef2207-0fe2-4e82-b0e8-60388620aaec admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:22.453989 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dec29b-4a7b-4da1-b4cc-26971fa11d1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:22.454433 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dec29b-4a7b-4da1-b4cc-26971fa11d1f service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:22.454775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dec29b-4a7b-4da1-b4cc-26971fa11d1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:22.455161 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dec29b-4a7b-4da1-b4cc-26971fa11d1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:22.465187 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dec29b-4a7b-4da1-b4cc-26971fa11d1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:22.467307 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dec29b-4a7b-4da1-b4cc-26971fa11d1f service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:22.477437 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1115/2224] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:15:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:15:23.760986 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4cd102-19bc-42ca-99a6-c6f5c70a22b7 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:23.760986 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4cd102-19bc-42ca-99a6-c6f5c70a22b7 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:23.760986 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4cd102-19bc-42ca-99a6-c6f5c70a22b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:23.806077 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-6b4cd102-19bc-42ca-99a6-c6f5c70a22b7 None None] MFA Rules not processed for user `18920ad97e6946df9689bd6d4abd50b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:15:23.807554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-6b4cd102-19bc-42ca-99a6-c6f5c70a22b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:23.929868 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1110/2225] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:15:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:15:23.942966 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dec29b-4a7b-4da1-b4cc-26971fa11d1f service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:23.945102 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-46dec29b-4a7b-4da1-b4cc-26971fa11d1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:24.044589 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dec29b-4a7b-4da1-b4cc-26971fa11d1f service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:24.051683 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a9c92-135c-43ee-9c16-fc2a519876d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:24.052650 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a9c92-135c-43ee-9c16-fc2a519876d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:24.052995 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a9c92-135c-43ee-9c16-fc2a519876d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:24.053348 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a9c92-135c-43ee-9c16-fc2a519876d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:24.075003 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26a9c92-135c-43ee-9c16-fc2a519876d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:24.075003 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26a9c92-135c-43ee-9c16-fc2a519876d8 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:24.078542 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1116/2226] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:15:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:15:25.486734 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36ed87-bc28-4da8-ae88-5535ccaa9b3e None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:25.487312 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36ed87-bc28-4da8-ae88-5535ccaa9b3e None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:25.487312 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36ed87-bc28-4da8-ae88-5535ccaa9b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:25.504430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:15:25.511478 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-4f36ed87-bc28-4da8-ae88-5535ccaa9b3e None None] MFA Rules not processed for user `18920ad97e6946df9689bd6d4abd50b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:15:25.513796 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-4f36ed87-bc28-4da8-ae88-5535ccaa9b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:25.524941 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1111/2227] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:15:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:15:30.297869 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a9c92-135c-43ee-9c16-fc2a519876d8 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:30.299553 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d26a9c92-135c-43ee-9c16-fc2a519876d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:30.311854 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a9c92-135c-43ee-9c16-fc2a519876d8 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:30.315637 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3669df91-8808-4c3f-ad11-3bdce02d23a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:30.316225 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3669df91-8808-4c3f-ad11-3bdce02d23a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:30.316507 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3669df91-8808-4c3f-ad11-3bdce02d23a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:30.316782 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3669df91-8808-4c3f-ad11-3bdce02d23a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:30.318443 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3669df91-8808-4c3f-ad11-3bdce02d23a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:30.379074 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3669df91-8808-4c3f-ad11-3bdce02d23a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:30.380577 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3669df91-8808-4c3f-ad11-3bdce02d23a6 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:30.389875 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1117/2228] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:15:30 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:15:31.341077 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36ed87-bc28-4da8-ae88-5535ccaa9b3e None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:31.344338 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-4f36ed87-bc28-4da8-ae88-5535ccaa9b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:31.496533 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36ed87-bc28-4da8-ae88-5535ccaa9b3e None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:31.504703 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ccac3-9209-4c6b-b83b-3991a0d74424 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:31.505475 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ccac3-9209-4c6b-b83b-3991a0d74424 service nova] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:31.506121 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ccac3-9209-4c6b-b83b-3991a0d74424 service nova] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:31.506482 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ccac3-9209-4c6b-b83b-3991a0d74424 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:31.508663 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-930ccac3-9209-4c6b-b83b-3991a0d74424 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:31.525243 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ccac3-9209-4c6b-b83b-3991a0d74424 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:31.527489 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ccac3-9209-4c6b-b83b-3991a0d74424 service nova] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:31.539536 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1112/2229] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:15:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:15:32.801572 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3669df91-8808-4c3f-ad11-3bdce02d23a6 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:32.816559 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3669df91-8808-4c3f-ad11-3bdce02d23a6 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:32.822511 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e92ee-33f0-49b8-8708-5f708c5f9b86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:32.823229 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e92ee-33f0-49b8-8708-5f708c5f9b86 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:32.823533 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e92ee-33f0-49b8-8708-5f708c5f9b86 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:32.823809 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e92ee-33f0-49b8-8708-5f708c5f9b86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:32.826270 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a25e92ee-33f0-49b8-8708-5f708c5f9b86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:32.840547 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e92ee-33f0-49b8-8708-5f708c5f9b86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:32.842239 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e92ee-33f0-49b8-8708-5f708c5f9b86 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:32.858678 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1118/2230] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:15:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:15:34.096310 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbac341-a07a-4cf6-8327-5c4ee8c3133a None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:34.096862 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbac341-a07a-4cf6-8327-5c4ee8c3133a None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:34.096862 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbac341-a07a-4cf6-8327-5c4ee8c3133a None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:34.098175 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1113/2231] 10.222.0.23 () {54 vars in 921 bytes} [Thu Oct 13 13:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:15:34.105900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9940dfb-2699-4354-9cbc-d6ee8aaab1a4 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:34.106444 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9940dfb-2699-4354-9cbc-d6ee8aaab1a4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:34.106444 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9940dfb-2699-4354-9cbc-d6ee8aaab1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:34.140125 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-a9940dfb-2699-4354-9cbc-d6ee8aaab1a4 None None] MFA Rules not processed for user `12700d985a724595af630c81e63f86d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:15:34.142134 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a9940dfb-2699-4354-9cbc-d6ee8aaab1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:34.168217 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1119/2232] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Oct 13 13:15:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:15:34.182307 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbac341-a07a-4cf6-8327-5c4ee8c3133a None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:34.184357 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-7cbac341-a07a-4cf6-8327-5c4ee8c3133a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:34.281434 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbac341-a07a-4cf6-8327-5c4ee8c3133a None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:34.286159 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c19ee8-3b9c-42e7-9c08-0986f885e46b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ce234c5b4643b2ac6341e0928f3830', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:34.287022 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c19ee8-3b9c-42e7-9c08-0986f885e46b service glance] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:34.288148 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c19ee8-3b9c-42e7-9c08-0986f885e46b service glance] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:34.288652 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c19ee8-3b9c-42e7-9c08-0986f885e46b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:34.302900 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c19ee8-3b9c-42e7-9c08-0986f885e46b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:34.304479 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c19ee8-3b9c-42e7-9c08-0986f885e46b service glance] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:34.317976 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1114/2233] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:15:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:15:34.390610 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d44543-390d-4d7f-8e89-3a994784ee91 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:34.391016 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d44543-390d-4d7f-8e89-3a994784ee91 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:34.391740 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d44543-390d-4d7f-8e89-3a994784ee91 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:34.393247 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1120/2234] 10.222.0.23 () {54 vars in 921 bytes} [Thu Oct 13 13:15:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:15:34.398345 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1049d4b-2487-4788-9442-5b0b094beb0c None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:34.398602 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1049d4b-2487-4788-9442-5b0b094beb0c None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:34.398783 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1049d4b-2487-4788-9442-5b0b094beb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:34.420570 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-d1049d4b-2487-4788-9442-5b0b094beb0c None None] MFA Rules not processed for user `846251a661c04d3f9f195c4ac09b4189`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:15:34.421785 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-d1049d4b-2487-4788-9442-5b0b094beb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:34.428762 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1115/2235] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Oct 13 13:15:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:15:34.441365 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d44543-390d-4d7f-8e89-3a994784ee91 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:34.453074 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d44543-390d-4d7f-8e89-3a994784ee91 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:34.456536 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8ce1b-6644-4597-bfec-dac4da54a4ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:34.457123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8ce1b-6644-4597-bfec-dac4da54a4ae service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:34.457466 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8ce1b-6644-4597-bfec-dac4da54a4ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:34.457720 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8ce1b-6644-4597-bfec-dac4da54a4ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:34.464472 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e8ce1b-6644-4597-bfec-dac4da54a4ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:34.466393 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e8ce1b-6644-4597-bfec-dac4da54a4ae service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:34.473941 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1121/2236] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:15:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:15:34.554985 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f53b2-2950-4809-8c57-a12a0c061582 None None] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:34.555252 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f53b2-2950-4809-8c57-a12a0c061582 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:34.555540 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f53b2-2950-4809-8c57-a12a0c061582 None None] PATH_INFO: `/` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:34.556914 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1116/2237] 10.222.0.23 () {54 vars in 921 bytes} [Thu Oct 13 13:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 13 13:15:34.561751 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68a07f-e42e-4469-b2cb-178a7a381028 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:34.562040 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68a07f-e42e-4469-b2cb-178a7a381028 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:34.562319 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68a07f-e42e-4469-b2cb-178a7a381028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:34.582017 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-ce68a07f-e42e-4469-b2cb-178a7a381028 None None] MFA Rules not processed for user `12700d985a724595af630c81e63f86d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:15:34.583260 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ce68a07f-e42e-4469-b2cb-178a7a381028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:34.593419 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1122/2238] 10.222.0.23 () {58 vars in 1018 bytes} [Thu Oct 13 13:15:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:15:34.603877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f53b2-2950-4809-8c57-a12a0c061582 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:34.614892 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f53b2-2950-4809-8c57-a12a0c061582 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:34.618672 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f912b-f5d2-4b80-a1d3-6ebf148406aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:34.619403 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f912b-f5d2-4b80-a1d3-6ebf148406aa service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:34.619778 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f912b-f5d2-4b80-a1d3-6ebf148406aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:34.620124 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f912b-f5d2-4b80-a1d3-6ebf148406aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:34.630379 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9f912b-f5d2-4b80-a1d3-6ebf148406aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:34.631818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9f912b-f5d2-4b80-a1d3-6ebf148406aa service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:34.643143 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1117/2239] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:15:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:15:35.522706 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68a07f-e42e-4469-b2cb-178a7a381028 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:35.538636 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68a07f-e42e-4469-b2cb-178a7a381028 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:35.544406 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f71af-dba7-4a7b-95af-4f995f1df59b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:35.545089 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f71af-dba7-4a7b-95af-4f995f1df59b service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:35.545437 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f71af-dba7-4a7b-95af-4f995f1df59b service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:35.545772 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f71af-dba7-4a7b-95af-4f995f1df59b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:35.548428 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4b6f71af-dba7-4a7b-95af-4f995f1df59b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:35.562428 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f71af-dba7-4a7b-95af-4f995f1df59b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:35.564044 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f71af-dba7-4a7b-95af-4f995f1df59b service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:35.574987 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1123/2240] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:15:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:15:36.693432 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24debab3-5c92-448e-bca3-26b6c2ddfe0f None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:36.693893 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24debab3-5c92-448e-bca3-26b6c2ddfe0f None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:36.693991 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24debab3-5c92-448e-bca3-26b6c2ddfe0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:36.712728 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-24debab3-5c92-448e-bca3-26b6c2ddfe0f None None] MFA Rules not processed for user `b6ae674f0efc445fa970fca64e13893a`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:15:36.713909 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-24debab3-5c92-448e-bca3-26b6c2ddfe0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:36.774627 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1118/2241] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Oct 13 13:15:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:15:36.786867 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f71af-dba7-4a7b-95af-4f995f1df59b service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:36.788271 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4b6f71af-dba7-4a7b-95af-4f995f1df59b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:36.802884 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:15:36.864633 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f71af-dba7-4a7b-95af-4f995f1df59b service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:36.876229 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f520a1d1-bad1-40ad-b2d1-1bf5c415924c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526000341fd146db8c661f1fd14d4801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:36.876891 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520a1d1-bad1-40ad-b2d1-1bf5c415924c service swift] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:36.877170 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520a1d1-bad1-40ad-b2d1-1bf5c415924c service swift] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:36.877442 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520a1d1-bad1-40ad-b2d1-1bf5c415924c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:36.886078 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f520a1d1-bad1-40ad-b2d1-1bf5c415924c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:36.888211 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f520a1d1-bad1-40ad-b2d1-1bf5c415924c service swift] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:36.896173 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1124/2242] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 13 13:15:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 13 13:15:46.767700 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=94 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:15:46.892960 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=26 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:15:50.021331 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30343b9f-07b4-4f25-baf0-db2c2cfe3e1c None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:50.021816 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30343b9f-07b4-4f25-baf0-db2c2cfe3e1c None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:50.021934 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30343b9f-07b4-4f25-baf0-db2c2cfe3e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:50.046683 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-30343b9f-07b4-4f25-baf0-db2c2cfe3e1c None None] MFA Rules not processed for user `18920ad97e6946df9689bd6d4abd50b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:15:50.048051 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-30343b9f-07b4-4f25-baf0-db2c2cfe3e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:50.055868 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1119/2243] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:15:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:15:54.627288 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3223e79-1891-4453-9185-78af7008958c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:54.627806 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3223e79-1891-4453-9185-78af7008958c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:54.627917 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3223e79-1891-4453-9185-78af7008958c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:54.651736 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c3223e79-1891-4453-9185-78af7008958c None None] MFA Rules not processed for user `b6ae674f0efc445fa970fca64e13893a`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:15:54.653456 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c3223e79-1891-4453-9185-78af7008958c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:15:54.663033 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1125/2244] 10.222.0.23 () {58 vars in 967 bytes} [Thu Oct 13 13:15:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:15:54.687322 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30343b9f-07b4-4f25-baf0-db2c2cfe3e1c None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:15:54.699016 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30343b9f-07b4-4f25-baf0-db2c2cfe3e1c None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:15:54.704753 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb83aa-5568-4279-80f1-07ffb71464ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526000341fd146db8c661f1fd14d4801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:15:54.705286 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb83aa-5568-4279-80f1-07ffb71464ce service swift] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:15:54.705586 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb83aa-5568-4279-80f1-07ffb71464ce service swift] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:15:54.705844 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb83aa-5568-4279-80f1-07ffb71464ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:15:54.714188 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb83aa-5568-4279-80f1-07ffb71464ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:15:54.716297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb83aa-5568-4279-80f1-07ffb71464ce service swift] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:15:54.724834 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1120/2245] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 13 13:15:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 13 13:16:03.067426 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3223e79-1891-4453-9185-78af7008958c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:03.077031 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3223e79-1891-4453-9185-78af7008958c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:03.080777 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97a107-e78c-4dd2-88c2-c9cd93690349 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:03.081379 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97a107-e78c-4dd2-88c2-c9cd93690349 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:03.081663 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97a107-e78c-4dd2-88c2-c9cd93690349 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:03.081988 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97a107-e78c-4dd2-88c2-c9cd93690349 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:03.083865 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0d97a107-e78c-4dd2-88c2-c9cd93690349 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:03.095803 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d97a107-e78c-4dd2-88c2-c9cd93690349 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:03.097229 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d97a107-e78c-4dd2-88c2-c9cd93690349 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:03.108075 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1126/2246] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:16:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:16:04.721025 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=9 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:16:07.294300 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb83aa-5568-4279-80f1-07ffb71464ce service swift] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:07.295739 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-75fb83aa-5568-4279-80f1-07ffb71464ce service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:07.310392 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb83aa-5568-4279-80f1-07ffb71464ce service swift] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:07.315313 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6ee64-e15d-45c8-b75f-16e1b61c0cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:07.315675 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6ee64-e15d-45c8-b75f-16e1b61c0cab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:07.315953 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6ee64-e15d-45c8-b75f-16e1b61c0cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:07.316219 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6ee64-e15d-45c8-b75f-16e1b61c0cab admin admin] PATH_INFO: `/v3/users/2e783caddcad473dbdd764919286e244` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:07.317483 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6ee64-e15d-45c8-b75f-16e1b61c0cab admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e783caddcad473dbdd764919286e244)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:07.319432 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6ee64-e15d-45c8-b75f-16e1b61c0cab admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:07.370968 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-06e6ee64-e15d-45c8-b75f-16e1b61c0cab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e783caddcad473dbdd764919286e244'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:07.378412 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-06e6ee64-e15d-45c8-b75f-16e1b61c0cab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e783caddcad473dbdd764919286e244'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:07.379160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-06e6ee64-e15d-45c8-b75f-16e1b61c0cab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e783caddcad473dbdd764919286e244'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:07.379787 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-06e6ee64-e15d-45c8-b75f-16e1b61c0cab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e783caddcad473dbdd764919286e244'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:07.389236 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-06e6ee64-e15d-45c8-b75f-16e1b61c0cab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e783caddcad473dbdd764919286e244'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:07.406864 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1121/2247] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:16:07 2022] DELETE /identity/v3/users/2e783caddcad473dbdd764919286e244 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:16:07.595458 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97a107-e78c-4dd2-88c2-c9cd93690349 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:07.611232 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97a107-e78c-4dd2-88c2-c9cd93690349 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:07.679219 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd7fdb-7500-4230-a9d0-9e7fc13cbc30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:07.679819 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd7fdb-7500-4230-a9d0-9e7fc13cbc30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:07.680106 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd7fdb-7500-4230-a9d0-9e7fc13cbc30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:07.680382 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd7fdb-7500-4230-a9d0-9e7fc13cbc30 admin admin] PATH_INFO: `/v3/projects/ecd3d5d2a3a54a07ba64f1fea4846c40` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:07.681537 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cd7fdb-7500-4230-a9d0-9e7fc13cbc30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecd3d5d2a3a54a07ba64f1fea4846c40)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:07.683473 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cd7fdb-7500-4230-a9d0-9e7fc13cbc30 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:07.731091 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b4cd7fdb-7500-4230-a9d0-9e7fc13cbc30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecd3d5d2a3a54a07ba64f1fea4846c40', 'request_id': 'req-b4cd7fdb-7500-4230-a9d0-9e7fc13cbc30'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:07.737303 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b4cd7fdb-7500-4230-a9d0-9e7fc13cbc30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecd3d5d2a3a54a07ba64f1fea4846c40', 'request_id': 'req-b4cd7fdb-7500-4230-a9d0-9e7fc13cbc30'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:07.744369 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b4cd7fdb-7500-4230-a9d0-9e7fc13cbc30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecd3d5d2a3a54a07ba64f1fea4846c40', 'request_id': 'req-b4cd7fdb-7500-4230-a9d0-9e7fc13cbc30'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:07.757445 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b4cd7fdb-7500-4230-a9d0-9e7fc13cbc30 admin admin] The token cache is being invalidate because project ecd3d5d2a3a54a07ba64f1fea4846c40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:16:07.757744 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b4cd7fdb-7500-4230-a9d0-9e7fc13cbc30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:07.758368 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b4cd7fdb-7500-4230-a9d0-9e7fc13cbc30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:07.759880 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1127/2248] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:16:07 2022] DELETE /identity/v3/projects/ecd3d5d2a3a54a07ba64f1fea4846c40 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:16:07.765889 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e85451-94d5-4205-ab32-2d8512945ce7 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:07.766152 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e85451-94d5-4205-ab32-2d8512945ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:07.766508 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e85451-94d5-4205-ab32-2d8512945ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:07.787128 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-e8e85451-94d5-4205-ab32-2d8512945ce7 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:16:07.788402 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-e8e85451-94d5-4205-ab32-2d8512945ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:07.845851 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1122/2249] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:16:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:16:07.854714 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711385ab-54f7-42f6-9975-f3511c30f0cf None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:07.855001 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711385ab-54f7-42f6-9975-f3511c30f0cf None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:07.855308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711385ab-54f7-42f6-9975-f3511c30f0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:07.875285 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-711385ab-54f7-42f6-9975-f3511c30f0cf None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:16:07.876476 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-711385ab-54f7-42f6-9975-f3511c30f0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:07.884093 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1128/2250] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:16:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:16:07.888748 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e85451-94d5-4205-ab32-2d8512945ce7 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:07.898372 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e85451-94d5-4205-ab32-2d8512945ce7 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:07.903864 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b173c1b-55f4-4ed9-8342-fc6b0ad41148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:07.904425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b173c1b-55f4-4ed9-8342-fc6b0ad41148 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:07.904735 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b173c1b-55f4-4ed9-8342-fc6b0ad41148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:07.909795 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b173c1b-55f4-4ed9-8342-fc6b0ad41148 admin admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:07.909795 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b173c1b-55f4-4ed9-8342-fc6b0ad41148 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:16:07.909795 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b173c1b-55f4-4ed9-8342-fc6b0ad41148 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:07.909795 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b173c1b-55f4-4ed9-8342-fc6b0ad41148 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:07.919107 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1123/2251] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:16:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:16:07.923447 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711385ab-54f7-42f6-9975-f3511c30f0cf None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:07.932884 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711385ab-54f7-42f6-9975-f3511c30f0cf None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:07.937975 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2012b3c3-b4bb-48ce-ac85-57d4cf536cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:07.938559 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012b3c3-b4bb-48ce-ac85-57d4cf536cf2 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:07.938837 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012b3c3-b4bb-48ce-ac85-57d4cf536cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:07.939129 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012b3c3-b4bb-48ce-ac85-57d4cf536cf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:07.939756 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2012b3c3-b4bb-48ce-ac85-57d4cf536cf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:07.941622 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2012b3c3-b4bb-48ce-ac85-57d4cf536cf2 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:07.967507 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1129/2252] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:16:07 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:16:07.971846 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b173c1b-55f4-4ed9-8342-fc6b0ad41148 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:07.981789 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b173c1b-55f4-4ed9-8342-fc6b0ad41148 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:08.037007 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f4094-4d7e-48c1-9ca4-1736699affe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:08.037625 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f4094-4d7e-48c1-9ca4-1736699affe5 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:08.037905 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f4094-4d7e-48c1-9ca4-1736699affe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:08.038206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f4094-4d7e-48c1-9ca4-1736699affe5 admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:08.038803 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f4094-4d7e-48c1-9ca4-1736699affe5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:08.040732 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f4094-4d7e-48c1-9ca4-1736699affe5 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:08.048739 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:16:08.063567 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1124/2253] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:16:07 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:16:08.067746 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2012b3c3-b4bb-48ce-ac85-57d4cf536cf2 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:08.078384 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2012b3c3-b4bb-48ce-ac85-57d4cf536cf2 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:08.082886 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e390f-7be8-4eba-9502-361efef91a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:08.083505 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e390f-7be8-4eba-9502-361efef91a87 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:08.083785 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e390f-7be8-4eba-9502-361efef91a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:08.084050 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e390f-7be8-4eba-9502-361efef91a87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:08.084686 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e390f-7be8-4eba-9502-361efef91a87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:08.086248 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e390f-7be8-4eba-9502-361efef91a87 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:08.096584 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1130/2254] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:16:08.100315 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f4094-4d7e-48c1-9ca4-1736699affe5 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:08.109963 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f4094-4d7e-48c1-9ca4-1736699affe5 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:08.114532 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5ac2a-c7e2-414a-9ccd-0ec46166e665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:08.115144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5ac2a-c7e2-414a-9ccd-0ec46166e665 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:08.115425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5ac2a-c7e2-414a-9ccd-0ec46166e665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:08.115695 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5ac2a-c7e2-414a-9ccd-0ec46166e665 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:08.116322 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e5ac2a-c7e2-414a-9ccd-0ec46166e665 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:08.117882 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e5ac2a-c7e2-414a-9ccd-0ec46166e665 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:08.128590 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1125/2255] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:16:08.132720 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e390f-7be8-4eba-9502-361efef91a87 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:08.142504 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e390f-7be8-4eba-9502-361efef91a87 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:08.147084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc433ad-da1b-4a45-a971-71725f7ddeff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:08.147740 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc433ad-da1b-4a45-a971-71725f7ddeff admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:08.148021 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc433ad-da1b-4a45-a971-71725f7ddeff admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:08.148296 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc433ad-da1b-4a45-a971-71725f7ddeff admin admin] PATH_INFO: `/v3/projects/41c4eb07ee1f4413a4e88dd4ce68d4ab/users/1f6783ab86fa47859ba0fb45ff427a06/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:08.163448 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc433ad-da1b-4a45-a971-71725f7ddeff admin admin] RBAC: Authorizing `identity:create_grant(project_id=41c4eb07ee1f4413a4e88dd4ce68d4ab, user_id=1f6783ab86fa47859ba0fb45ff427a06, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:08.166016 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc433ad-da1b-4a45-a971-71725f7ddeff admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:08.179007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1131/2256] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:16:08 2022] PUT /identity/v3/projects/41c4eb07ee1f4413a4e88dd4ce68d4ab/users/1f6783ab86fa47859ba0fb45ff427a06/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:16:08.189007 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744f29e-301d-4544-9ea0-5e74f589d8d2 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:08.189364 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744f29e-301d-4544-9ea0-5e74f589d8d2 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:08.189634 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744f29e-301d-4544-9ea0-5e74f589d8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:08.217117 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-d744f29e-301d-4544-9ea0-5e74f589d8d2 None None] MFA Rules not processed for user `1f6783ab86fa47859ba0fb45ff427a06`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:16:08.218315 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-d744f29e-301d-4544-9ea0-5e74f589d8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:08.280816 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1126/2257] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:16:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:16:08.291417 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc433ad-da1b-4a45-a971-71725f7ddeff admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:08.293115 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4cc433ad-da1b-4a45-a971-71725f7ddeff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:08.303841 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:16:08.368012 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc433ad-da1b-4a45-a971-71725f7ddeff admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:08.373491 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fcd3e-906a-4294-9576-d60087bdbb61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:08.374110 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fcd3e-906a-4294-9576-d60087bdbb61 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:08.374390 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fcd3e-906a-4294-9576-d60087bdbb61 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:08.374661 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fcd3e-906a-4294-9576-d60087bdbb61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:08.382157 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63fcd3e-906a-4294-9576-d60087bdbb61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:08.383762 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63fcd3e-906a-4294-9576-d60087bdbb61 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:08.392764 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1132/2258] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:16:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:16:11.572674 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7cb4b-564f-44b8-a2dd-5b73b3d3ef98 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:11.573150 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7cb4b-564f-44b8-a2dd-5b73b3d3ef98 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:11.573258 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7cb4b-564f-44b8-a2dd-5b73b3d3ef98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:11.591955 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-30a7cb4b-564f-44b8-a2dd-5b73b3d3ef98 None None] MFA Rules not processed for user `b6ae674f0efc445fa970fca64e13893a`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:16:11.593141 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-30a7cb4b-564f-44b8-a2dd-5b73b3d3ef98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:11.647928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1127/2259] 10.222.0.23 () {58 vars in 1009 bytes} [Thu Oct 13 13:16:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:16:11.661145 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fcd3e-906a-4294-9576-d60087bdbb61 service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:11.662613 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a63fcd3e-906a-4294-9576-d60087bdbb61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:11.730844 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fcd3e-906a-4294-9576-d60087bdbb61 service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:11.736397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4e30f-cff4-4a5b-972e-b6068e2a683b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526000341fd146db8c661f1fd14d4801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:11.736990 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4e30f-cff4-4a5b-972e-b6068e2a683b service swift] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:11.737267 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4e30f-cff4-4a5b-972e-b6068e2a683b service swift] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:11.737533 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4e30f-cff4-4a5b-972e-b6068e2a683b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:11.745515 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c4e30f-cff4-4a5b-972e-b6068e2a683b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:11.747106 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c4e30f-cff4-4a5b-972e-b6068e2a683b service swift] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:11.755000 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1133/2260] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 13 13:16:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 13 13:16:19.007567 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7cb4b-564f-44b8-a2dd-5b73b3d3ef98 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:19.092173 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7cb4b-564f-44b8-a2dd-5b73b3d3ef98 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:19.093824 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2492bda-dd42-4562-8a05-e64421ccdedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:19.094529 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2492bda-dd42-4562-8a05-e64421ccdedc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:19.094902 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2492bda-dd42-4562-8a05-e64421ccdedc admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:19.095362 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2492bda-dd42-4562-8a05-e64421ccdedc admin admin] PATH_INFO: `/v3/users/1f6783ab86fa47859ba0fb45ff427a06` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:19.096784 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2492bda-dd42-4562-8a05-e64421ccdedc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f6783ab86fa47859ba0fb45ff427a06)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:19.099217 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2492bda-dd42-4562-8a05-e64421ccdedc admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:19.117930 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:INSERT=3,keystone:SELECT=96,keystone:DELETE=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:16:19.169621 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-d2492bda-dd42-4562-8a05-e64421ccdedc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f6783ab86fa47859ba0fb45ff427a06'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:19.178828 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-d2492bda-dd42-4562-8a05-e64421ccdedc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f6783ab86fa47859ba0fb45ff427a06'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:19.179595 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-d2492bda-dd42-4562-8a05-e64421ccdedc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f6783ab86fa47859ba0fb45ff427a06'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:19.180475 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-d2492bda-dd42-4562-8a05-e64421ccdedc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f6783ab86fa47859ba0fb45ff427a06'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:19.189580 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-d2492bda-dd42-4562-8a05-e64421ccdedc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f6783ab86fa47859ba0fb45ff427a06'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:19.209590 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1128/2261] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:16:19 2022] DELETE /identity/v3/users/1f6783ab86fa47859ba0fb45ff427a06 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:16:19.216901 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43284901-42b9-4b63-aec7-aacd38968bca None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:19.217420 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43284901-42b9-4b63-aec7-aacd38968bca None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:19.218131 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43284901-42b9-4b63-aec7-aacd38968bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:19.243843 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-43284901-42b9-4b63-aec7-aacd38968bca None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:16:19.245268 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-43284901-42b9-4b63-aec7-aacd38968bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:19.329941 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1134/2262] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:16:19.338034 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2492bda-dd42-4562-8a05-e64421ccdedc admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:19.339633 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-d2492bda-dd42-4562-8a05-e64421ccdedc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:19.411198 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2492bda-dd42-4562-8a05-e64421ccdedc admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:19.414798 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c529e-0cb6-468c-a389-4c137afe8b55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:19.415480 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c529e-0cb6-468c-a389-4c137afe8b55 service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:19.415756 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c529e-0cb6-468c-a389-4c137afe8b55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:19.416030 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c529e-0cb6-468c-a389-4c137afe8b55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:19.427843 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c529e-0cb6-468c-a389-4c137afe8b55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:19.429320 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c529e-0cb6-468c-a389-4c137afe8b55 service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:19.440427 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1129/2263] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:16:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:16:19.761430 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43284901-42b9-4b63-aec7-aacd38968bca None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:19.763721 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-43284901-42b9-4b63-aec7-aacd38968bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:19.782014 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43284901-42b9-4b63-aec7-aacd38968bca None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:19.788566 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b614e-bb3d-4387-b5bf-2dae96262aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:19.789387 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b614e-bb3d-4387-b5bf-2dae96262aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:19.789670 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b614e-bb3d-4387-b5bf-2dae96262aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:19.789943 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b614e-bb3d-4387-b5bf-2dae96262aee admin admin] PATH_INFO: `/v3/projects/41c4eb07ee1f4413a4e88dd4ce68d4ab` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:19.791182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b614e-bb3d-4387-b5bf-2dae96262aee admin admin] RBAC: Authorizing `identity:delete_project(project_id=41c4eb07ee1f4413a4e88dd4ce68d4ab)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:19.793280 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b614e-bb3d-4387-b5bf-2dae96262aee admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:19.824004 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:16:19.860473 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-de9b614e-bb3d-4387-b5bf-2dae96262aee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41c4eb07ee1f4413a4e88dd4ce68d4ab', 'request_id': 'req-de9b614e-bb3d-4387-b5bf-2dae96262aee'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:19.866900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-de9b614e-bb3d-4387-b5bf-2dae96262aee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41c4eb07ee1f4413a4e88dd4ce68d4ab', 'request_id': 'req-de9b614e-bb3d-4387-b5bf-2dae96262aee'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:19.874236 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-de9b614e-bb3d-4387-b5bf-2dae96262aee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41c4eb07ee1f4413a4e88dd4ce68d4ab', 'request_id': 'req-de9b614e-bb3d-4387-b5bf-2dae96262aee'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:19.891413 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-de9b614e-bb3d-4387-b5bf-2dae96262aee admin admin] The token cache is being invalidate because project 41c4eb07ee1f4413a4e88dd4ce68d4ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:16:19.891886 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-de9b614e-bb3d-4387-b5bf-2dae96262aee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:19.892654 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-de9b614e-bb3d-4387-b5bf-2dae96262aee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:19.894380 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1135/2264] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:16:19 2022] DELETE /identity/v3/projects/41c4eb07ee1f4413a4e88dd4ce68d4ab => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:16:19.899438 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0977a8-c48d-48f5-adb9-59d9bb7145ad None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:19.899873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0977a8-c48d-48f5-adb9-59d9bb7145ad None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:19.900254 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0977a8-c48d-48f5-adb9-59d9bb7145ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:19.923006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-ee0977a8-c48d-48f5-adb9-59d9bb7145ad None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:16:19.924570 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-ee0977a8-c48d-48f5-adb9-59d9bb7145ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:19.990092 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1130/2265] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:16:19.998288 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfeb82a-71ab-41cd-ae79-0e5006ac1cc8 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:19.998597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfeb82a-71ab-41cd-ae79-0e5006ac1cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:19.998880 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfeb82a-71ab-41cd-ae79-0e5006ac1cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:20.021158 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-dbfeb82a-71ab-41cd-ae79-0e5006ac1cc8 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:16:20.022416 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-dbfeb82a-71ab-41cd-ae79-0e5006ac1cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:20.030460 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1136/2266] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:16:20.035931 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0977a8-c48d-48f5-adb9-59d9bb7145ad None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:20.046488 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0977a8-c48d-48f5-adb9-59d9bb7145ad None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:20.051781 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ccde96-6ab9-4699-adc1-505f1c90aad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:20.052587 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ccde96-6ab9-4699-adc1-505f1c90aad1 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:20.052942 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ccde96-6ab9-4699-adc1-505f1c90aad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:20.053396 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ccde96-6ab9-4699-adc1-505f1c90aad1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:20.054162 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ccde96-6ab9-4699-adc1-505f1c90aad1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:16:20.054571 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ccde96-6ab9-4699-adc1-505f1c90aad1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:20.056327 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ccde96-6ab9-4699-adc1-505f1c90aad1 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:20.075127 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1131/2267] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:16:20.078364 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfeb82a-71ab-41cd-ae79-0e5006ac1cc8 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:20.089395 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfeb82a-71ab-41cd-ae79-0e5006ac1cc8 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:20.095118 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96fd25-72b7-4c55-9696-74ac538dc694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:20.095831 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96fd25-72b7-4c55-9696-74ac538dc694 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:20.096144 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96fd25-72b7-4c55-9696-74ac538dc694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:20.096488 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96fd25-72b7-4c55-9696-74ac538dc694 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:20.097191 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da96fd25-72b7-4c55-9696-74ac538dc694 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:20.099187 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da96fd25-72b7-4c55-9696-74ac538dc694 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:20.130820 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1137/2268] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:16:20 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:16:20.136066 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ccde96-6ab9-4699-adc1-505f1c90aad1 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:20.148487 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ccde96-6ab9-4699-adc1-505f1c90aad1 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:20.175400 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:16:20.227186 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28812c88-be7e-4768-a90b-8e7ed7cca8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:20.228214 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28812c88-be7e-4768-a90b-8e7ed7cca8f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:20.228704 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28812c88-be7e-4768-a90b-8e7ed7cca8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:20.229182 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28812c88-be7e-4768-a90b-8e7ed7cca8f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:20.230024 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28812c88-be7e-4768-a90b-8e7ed7cca8f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:20.232655 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28812c88-be7e-4768-a90b-8e7ed7cca8f4 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:20.258053 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1132/2269] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:16:20 2022] POST /identity/v3/users => generated 288 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:16:20.262446 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96fd25-72b7-4c55-9696-74ac538dc694 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:20.273384 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96fd25-72b7-4c55-9696-74ac538dc694 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:20.278702 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35971af9-a430-48d5-b775-1d3a36e3e1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:20.279390 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35971af9-a430-48d5-b775-1d3a36e3e1bc admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:20.279683 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35971af9-a430-48d5-b775-1d3a36e3e1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:20.280007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35971af9-a430-48d5-b775-1d3a36e3e1bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:20.280630 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35971af9-a430-48d5-b775-1d3a36e3e1bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:20.282279 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35971af9-a430-48d5-b775-1d3a36e3e1bc admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:20.297439 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1138/2270] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:16:20.299255 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28812c88-be7e-4768-a90b-8e7ed7cca8f4 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:20.310662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28812c88-be7e-4768-a90b-8e7ed7cca8f4 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:20.316166 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba3ce3e-9936-4911-9d5d-d058267284ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:20.316902 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba3ce3e-9936-4911-9d5d-d058267284ff admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:20.317951 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba3ce3e-9936-4911-9d5d-d058267284ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:20.318337 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba3ce3e-9936-4911-9d5d-d058267284ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:20.319131 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba3ce3e-9936-4911-9d5d-d058267284ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:20.320848 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba3ce3e-9936-4911-9d5d-d058267284ff admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:20.332663 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1133/2271] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:16:20.336802 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35971af9-a430-48d5-b775-1d3a36e3e1bc admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:20.348324 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35971af9-a430-48d5-b775-1d3a36e3e1bc admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:20.353535 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7b081-8a3f-4466-9821-4b7f8f049073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:20.354219 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7b081-8a3f-4466-9821-4b7f8f049073 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:20.354821 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7b081-8a3f-4466-9821-4b7f8f049073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:20.354821 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7b081-8a3f-4466-9821-4b7f8f049073 admin admin] PATH_INFO: `/v3/projects/3a5a12da1f8944bb9e447fb28ee20e66/users/2ae75f5b6c70464696d5ecba26bbc8e7/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:20.373376 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7b081-8a3f-4466-9821-4b7f8f049073 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a5a12da1f8944bb9e447fb28ee20e66, user_id=2ae75f5b6c70464696d5ecba26bbc8e7, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:20.376136 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7b081-8a3f-4466-9821-4b7f8f049073 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:20.393394 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1139/2272] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:16:20 2022] PUT /identity/v3/projects/3a5a12da1f8944bb9e447fb28ee20e66/users/2ae75f5b6c70464696d5ecba26bbc8e7/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:16:20.405807 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552daf27-33c4-4478-9225-94a296c34118 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:20.406135 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552daf27-33c4-4478-9225-94a296c34118 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:20.406407 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552daf27-33c4-4478-9225-94a296c34118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:20.438749 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-552daf27-33c4-4478-9225-94a296c34118 None None] MFA Rules not processed for user `2ae75f5b6c70464696d5ecba26bbc8e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:16:20.440055 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-552daf27-33c4-4478-9225-94a296c34118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:20.526991 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1134/2273] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:16:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:16:20.531836 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7b081-8a3f-4466-9821-4b7f8f049073 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:20.602898 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7b081-8a3f-4466-9821-4b7f8f049073 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:20.608829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf39ee-33ad-4d47-baaf-ace652573bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:20.609457 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf39ee-33ad-4d47-baaf-ace652573bfc admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:20.609728 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf39ee-33ad-4d47-baaf-ace652573bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:20.610005 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf39ee-33ad-4d47-baaf-ace652573bfc admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:20.610715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf39ee-33ad-4d47-baaf-ace652573bfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:20.612684 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf39ee-33ad-4d47-baaf-ace652573bfc admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:20.647048 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1140/2274] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:16:20 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:16:20.648275 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552daf27-33c4-4478-9225-94a296c34118 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:20.661373 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552daf27-33c4-4478-9225-94a296c34118 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:20.736393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d5e8d-3dda-47c2-a751-202edd04fe35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:20.737467 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d5e8d-3dda-47c2-a751-202edd04fe35 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:20.737800 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d5e8d-3dda-47c2-a751-202edd04fe35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:20.738028 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d5e8d-3dda-47c2-a751-202edd04fe35 admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:20.738709 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d5e8d-3dda-47c2-a751-202edd04fe35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:20.740829 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d5e8d-3dda-47c2-a751-202edd04fe35 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:20.765925 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1135/2275] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:16:20 2022] POST /identity/v3/users => generated 294 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:16:20.770291 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf39ee-33ad-4d47-baaf-ace652573bfc admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:20.782667 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf39ee-33ad-4d47-baaf-ace652573bfc admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:20.788201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef6e74-040b-4da3-af38-ee335d4ab696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:20.788927 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef6e74-040b-4da3-af38-ee335d4ab696 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:20.789308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef6e74-040b-4da3-af38-ee335d4ab696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:20.789677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef6e74-040b-4da3-af38-ee335d4ab696 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:20.790430 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef6e74-040b-4da3-af38-ee335d4ab696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:20.792172 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef6e74-040b-4da3-af38-ee335d4ab696 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:20.808872 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1141/2276] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:16:20.810092 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d5e8d-3dda-47c2-a751-202edd04fe35 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:20.820903 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d5e8d-3dda-47c2-a751-202edd04fe35 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:20.826547 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec4305-1173-4c69-8e6a-69180b99372f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:20.827317 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec4305-1173-4c69-8e6a-69180b99372f admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:20.827667 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec4305-1173-4c69-8e6a-69180b99372f admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:20.828049 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec4305-1173-4c69-8e6a-69180b99372f admin admin] PATH_INFO: `/v3/projects/4d65446ad1ff462ebbddb400362dfc1d/users/24f63cf37354457e8f6b689795de418d/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:20.850919 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec4305-1173-4c69-8e6a-69180b99372f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d65446ad1ff462ebbddb400362dfc1d, user_id=24f63cf37354457e8f6b689795de418d, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:20.854019 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec4305-1173-4c69-8e6a-69180b99372f admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:20.858083 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:16:20.870614 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1136/2277] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:16:20 2022] PUT /identity/v3/projects/4d65446ad1ff462ebbddb400362dfc1d/users/24f63cf37354457e8f6b689795de418d/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:16:20.874709 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef6e74-040b-4da3-af38-ee335d4ab696 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:20.886521 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef6e74-040b-4da3-af38-ee335d4ab696 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:20.911637 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:16:20.972936 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab0755-1fae-473c-a538-29a9e1485408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:20.973543 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab0755-1fae-473c-a538-29a9e1485408 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:20.973818 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab0755-1fae-473c-a538-29a9e1485408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:20.974083 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab0755-1fae-473c-a538-29a9e1485408 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:20.974717 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab0755-1fae-473c-a538-29a9e1485408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:20.976664 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab0755-1fae-473c-a538-29a9e1485408 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:20.994406 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1142/2278] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:16:20.998646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec4305-1173-4c69-8e6a-69180b99372f admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:21.010945 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec4305-1173-4c69-8e6a-69180b99372f admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:21.016426 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aaace3-4ed4-449a-ac09-81c50b3a6415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:21.017211 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaace3-4ed4-449a-ac09-81c50b3a6415 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:21.017595 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaace3-4ed4-449a-ac09-81c50b3a6415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:21.017973 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaace3-4ed4-449a-ac09-81c50b3a6415 admin admin] PATH_INFO: `/v3/projects/4d65446ad1ff462ebbddb400362dfc1d/users/24f63cf37354457e8f6b689795de418d/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:21.020172 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aaace3-4ed4-449a-ac09-81c50b3a6415 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d65446ad1ff462ebbddb400362dfc1d, user_id=24f63cf37354457e8f6b689795de418d, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:21.022933 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aaace3-4ed4-449a-ac09-81c50b3a6415 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:21.037607 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1137/2279] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:16:20 2022] PUT /identity/v3/projects/4d65446ad1ff462ebbddb400362dfc1d/users/24f63cf37354457e8f6b689795de418d/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:16:21.050033 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca9bf4-e923-4d48-bdc6-ab79c6deb3c9 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:21.050412 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca9bf4-e923-4d48-bdc6-ab79c6deb3c9 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:21.050686 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca9bf4-e923-4d48-bdc6-ab79c6deb3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:21.099818 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-cbca9bf4-e923-4d48-bdc6-ab79c6deb3c9 None None] MFA Rules not processed for user `24f63cf37354457e8f6b689795de418d`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:16:21.103445 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-cbca9bf4-e923-4d48-bdc6-ab79c6deb3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:21.191350 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1143/2280] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:16:21 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:16:21.203769 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aaace3-4ed4-449a-ac09-81c50b3a6415 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:21.205456 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-79aaace3-4ed4-449a-ac09-81c50b3a6415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:21.273746 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aaace3-4ed4-449a-ac09-81c50b3a6415 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:21.280545 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc884422-19a6-4509-aaf3-21186d15e97f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:21.281751 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc884422-19a6-4509-aaf3-21186d15e97f service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:21.282080 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc884422-19a6-4509-aaf3-21186d15e97f service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:21.282355 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc884422-19a6-4509-aaf3-21186d15e97f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:21.357584 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc884422-19a6-4509-aaf3-21186d15e97f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:21.359199 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc884422-19a6-4509-aaf3-21186d15e97f service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:21.369136 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1138/2281] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:16:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:16:26.329975 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873189f0-c0e9-463d-8234-7c20607ef38c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:26.330445 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873189f0-c0e9-463d-8234-7c20607ef38c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:26.330546 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873189f0-c0e9-463d-8234-7c20607ef38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:26.349226 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-873189f0-c0e9-463d-8234-7c20607ef38c None None] MFA Rules not processed for user `b6ae674f0efc445fa970fca64e13893a`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:16:26.350412 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-873189f0-c0e9-463d-8234-7c20607ef38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:26.403386 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1144/2282] 10.222.0.23 () {58 vars in 967 bytes} [Thu Oct 13 13:16:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:16:26.415285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc884422-19a6-4509-aaf3-21186d15e97f service cinder] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:26.416660 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-bc884422-19a6-4509-aaf3-21186d15e97f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:26.484268 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc884422-19a6-4509-aaf3-21186d15e97f service cinder] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:26.489159 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd7c28-a7fa-4eee-a8d3-45feaa79d0fc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '526000341fd146db8c661f1fd14d4801', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:26.489751 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd7c28-a7fa-4eee-a8d3-45feaa79d0fc service swift] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:26.490030 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd7c28-a7fa-4eee-a8d3-45feaa79d0fc service swift] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:26.490306 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd7c28-a7fa-4eee-a8d3-45feaa79d0fc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:26.499544 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd7c28-a7fa-4eee-a8d3-45feaa79d0fc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:26.501148 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd7c28-a7fa-4eee-a8d3-45feaa79d0fc service swift] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:26.509132 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1139/2283] 10.222.0.23 () {60 vars in 1384 bytes} [Thu Oct 13 13:16:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 13 13:16:27.179618 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873189f0-c0e9-463d-8234-7c20607ef38c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:27.188816 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873189f0-c0e9-463d-8234-7c20607ef38c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:27.192379 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed45c7b-3ce8-49d1-8165-a311356377c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:27.192968 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed45c7b-3ce8-49d1-8165-a311356377c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:27.193247 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed45c7b-3ce8-49d1-8165-a311356377c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:27.193519 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed45c7b-3ce8-49d1-8165-a311356377c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:27.202709 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed45c7b-3ce8-49d1-8165-a311356377c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:27.204155 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed45c7b-3ce8-49d1-8165-a311356377c1 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:27.213400 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1145/2284] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:16:27 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:16:28.411548 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd7c28-a7fa-4eee-a8d3-45feaa79d0fc service swift] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:28.412959 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-e3fd7c28-a7fa-4eee-a8d3-45feaa79d0fc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:28.444038 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:16:28.478692 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd7c28-a7fa-4eee-a8d3-45feaa79d0fc service swift] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:28.483651 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a9e4a-efdd-4480-8d16-17d189ac2099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:28.484267 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a9e4a-efdd-4480-8d16-17d189ac2099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:28.484544 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a9e4a-efdd-4480-8d16-17d189ac2099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:28.484809 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a9e4a-efdd-4480-8d16-17d189ac2099 admin admin] PATH_INFO: `/v3/users/18920ad97e6946df9689bd6d4abd50b0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:28.485908 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a9e4a-efdd-4480-8d16-17d189ac2099 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18920ad97e6946df9689bd6d4abd50b0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:28.487855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a9e4a-efdd-4480-8d16-17d189ac2099 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:28.533039 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-5b7a9e4a-efdd-4480-8d16-17d189ac2099 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18920ad97e6946df9689bd6d4abd50b0'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:28.540113 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-5b7a9e4a-efdd-4480-8d16-17d189ac2099 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18920ad97e6946df9689bd6d4abd50b0'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:28.540819 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-5b7a9e4a-efdd-4480-8d16-17d189ac2099 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18920ad97e6946df9689bd6d4abd50b0'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:28.541508 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-5b7a9e4a-efdd-4480-8d16-17d189ac2099 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18920ad97e6946df9689bd6d4abd50b0'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:28.550814 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-5b7a9e4a-efdd-4480-8d16-17d189ac2099 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18920ad97e6946df9689bd6d4abd50b0'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:28.569200 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1140/2285] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:16:28 2022] DELETE /identity/v3/users/18920ad97e6946df9689bd6d4abd50b0 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:16:28.742829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed45c7b-3ce8-49d1-8165-a311356377c1 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:28.755347 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed45c7b-3ce8-49d1-8165-a311356377c1 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:28.783198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:16:28.812812 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b203c-509f-405e-9bf9-3353c582a050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:28.813431 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b203c-509f-405e-9bf9-3353c582a050 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:28.813708 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b203c-509f-405e-9bf9-3353c582a050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:28.813976 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b203c-509f-405e-9bf9-3353c582a050 admin admin] PATH_INFO: `/v3/projects/12163a99662b48759b3fd7da51445ecf` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:28.815065 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b203c-509f-405e-9bf9-3353c582a050 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12163a99662b48759b3fd7da51445ecf)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:28.816961 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b203c-509f-405e-9bf9-3353c582a050 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:28.861183 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-176b203c-509f-405e-9bf9-3353c582a050 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12163a99662b48759b3fd7da51445ecf', 'request_id': 'req-176b203c-509f-405e-9bf9-3353c582a050'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:28.866745 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-176b203c-509f-405e-9bf9-3353c582a050 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12163a99662b48759b3fd7da51445ecf', 'request_id': 'req-176b203c-509f-405e-9bf9-3353c582a050'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:28.872743 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-176b203c-509f-405e-9bf9-3353c582a050 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12163a99662b48759b3fd7da51445ecf', 'request_id': 'req-176b203c-509f-405e-9bf9-3353c582a050'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:28.884625 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-176b203c-509f-405e-9bf9-3353c582a050 admin admin] The token cache is being invalidate because project 12163a99662b48759b3fd7da51445ecf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:16:28.884917 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-176b203c-509f-405e-9bf9-3353c582a050 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:28.885536 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-176b203c-509f-405e-9bf9-3353c582a050 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:28.887594 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1146/2286] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:16:28 2022] DELETE /identity/v3/projects/12163a99662b48759b3fd7da51445ecf => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:16:28.892633 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c6598e-a2c6-4cdf-af87-a88873ef87b7 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:28.892939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c6598e-a2c6-4cdf-af87-a88873ef87b7 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:28.893206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c6598e-a2c6-4cdf-af87-a88873ef87b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:28.913308 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-04c6598e-a2c6-4cdf-af87-a88873ef87b7 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:16:28.914503 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-04c6598e-a2c6-4cdf-af87-a88873ef87b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:28.972101 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1141/2287] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:16:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:16:28.980298 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e075f7-09a0-4a5e-a43c-0ca387482fda None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:28.980586 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e075f7-09a0-4a5e-a43c-0ca387482fda None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:28.980891 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e075f7-09a0-4a5e-a43c-0ca387482fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:29.000440 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-77e075f7-09a0-4a5e-a43c-0ca387482fda None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:16:29.001567 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-77e075f7-09a0-4a5e-a43c-0ca387482fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:29.010658 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1147/2288] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:16:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:16:29.015430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c6598e-a2c6-4cdf-af87-a88873ef87b7 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:29.025160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c6598e-a2c6-4cdf-af87-a88873ef87b7 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:29.029700 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad2e87-0aec-4803-a654-08c0bdcf8081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:29.030293 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad2e87-0aec-4803-a654-08c0bdcf8081 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:29.030573 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad2e87-0aec-4803-a654-08c0bdcf8081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:29.030846 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad2e87-0aec-4803-a654-08c0bdcf8081 admin admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:29.031520 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad2e87-0aec-4803-a654-08c0bdcf8081 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:16:29.031829 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad2e87-0aec-4803-a654-08c0bdcf8081 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:29.033433 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad2e87-0aec-4803-a654-08c0bdcf8081 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:29.044835 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1142/2289] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:16:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:16:29.049360 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e075f7-09a0-4a5e-a43c-0ca387482fda None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:29.058378 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e075f7-09a0-4a5e-a43c-0ca387482fda None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:29.063482 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481d013-3a62-4281-9cb1-6463da8f05ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:29.064068 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481d013-3a62-4281-9cb1-6463da8f05ec admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:29.064347 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481d013-3a62-4281-9cb1-6463da8f05ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:29.064619 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481d013-3a62-4281-9cb1-6463da8f05ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:29.065243 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3481d013-3a62-4281-9cb1-6463da8f05ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:29.067148 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3481d013-3a62-4281-9cb1-6463da8f05ec admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:29.093601 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1148/2290] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:16:29 2022] POST /identity/v3/projects => generated 367 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:16:29.097673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad2e87-0aec-4803-a654-08c0bdcf8081 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:29.107738 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad2e87-0aec-4803-a654-08c0bdcf8081 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:29.136834 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:16:29.164042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8ffdf-6642-4242-9db3-25692557e010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:29.164661 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8ffdf-6642-4242-9db3-25692557e010 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:29.164942 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8ffdf-6642-4242-9db3-25692557e010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:29.165213 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8ffdf-6642-4242-9db3-25692557e010 admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:29.165836 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8ffdf-6642-4242-9db3-25692557e010 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:29.167775 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8ffdf-6642-4242-9db3-25692557e010 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:29.189309 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1143/2291] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:16:29 2022] POST /identity/v3/users => generated 284 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:16:29.193525 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481d013-3a62-4281-9cb1-6463da8f05ec admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:29.204367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481d013-3a62-4281-9cb1-6463da8f05ec admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:29.208950 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799f4e8-a836-45be-83c9-64f46b01eb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:29.209542 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799f4e8-a836-45be-83c9-64f46b01eb77 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:29.209818 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799f4e8-a836-45be-83c9-64f46b01eb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:29.210094 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799f4e8-a836-45be-83c9-64f46b01eb77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:29.210729 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5799f4e8-a836-45be-83c9-64f46b01eb77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:29.212361 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5799f4e8-a836-45be-83c9-64f46b01eb77 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:29.222153 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1149/2292] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:16:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:16:29.225964 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8ffdf-6642-4242-9db3-25692557e010 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:29.235773 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8ffdf-6642-4242-9db3-25692557e010 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:29.240663 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9d50a-5c57-493d-a6a8-b0d5d03fbfbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:29.241263 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9d50a-5c57-493d-a6a8-b0d5d03fbfbf admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:29.241614 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9d50a-5c57-493d-a6a8-b0d5d03fbfbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:29.241810 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9d50a-5c57-493d-a6a8-b0d5d03fbfbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:29.242454 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac9d50a-5c57-493d-a6a8-b0d5d03fbfbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:29.244082 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac9d50a-5c57-493d-a6a8-b0d5d03fbfbf admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:29.255387 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1144/2293] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:16:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:16:29.259420 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799f4e8-a836-45be-83c9-64f46b01eb77 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:29.269862 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799f4e8-a836-45be-83c9-64f46b01eb77 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:29.274517 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54d9f8-f225-4a92-9765-9490782b51c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:29.275197 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54d9f8-f225-4a92-9765-9490782b51c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:29.275482 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54d9f8-f225-4a92-9765-9490782b51c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:29.275753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54d9f8-f225-4a92-9765-9490782b51c9 admin admin] PATH_INFO: `/v3/projects/7913b472c713462daa5945cd12c82cf6/users/61c90e0f6600441883fc81ea1650e7b2/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:29.289967 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce54d9f8-f225-4a92-9765-9490782b51c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7913b472c713462daa5945cd12c82cf6, user_id=61c90e0f6600441883fc81ea1650e7b2, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:29.292730 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce54d9f8-f225-4a92-9765-9490782b51c9 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:29.306306 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1150/2294] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:16:29 2022] PUT /identity/v3/projects/7913b472c713462daa5945cd12c82cf6/users/61c90e0f6600441883fc81ea1650e7b2/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:16:29.316879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597bf22-5d9b-4fa0-b8bc-0efe3a607749 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:29.317171 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597bf22-5d9b-4fa0-b8bc-0efe3a607749 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:29.317448 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597bf22-5d9b-4fa0-b8bc-0efe3a607749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:29.344555 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-d597bf22-5d9b-4fa0-b8bc-0efe3a607749 None None] MFA Rules not processed for user `61c90e0f6600441883fc81ea1650e7b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:16:29.345785 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-d597bf22-5d9b-4fa0-b8bc-0efe3a607749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:29.407782 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1145/2295] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:16:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:16:29.419871 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54d9f8-f225-4a92-9765-9490782b51c9 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:29.421563 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ce54d9f8-f225-4a92-9765-9490782b51c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:29.477690 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54d9f8-f225-4a92-9765-9490782b51c9 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:29.481892 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f40043-f41e-4642-8987-75a3bc95ec6e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ce234c5b4643b2ac6341e0928f3830', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:29.482489 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f40043-f41e-4642-8987-75a3bc95ec6e service glance] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:29.482774 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f40043-f41e-4642-8987-75a3bc95ec6e service glance] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:29.483048 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f40043-f41e-4642-8987-75a3bc95ec6e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:29.491185 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f40043-f41e-4642-8987-75a3bc95ec6e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:29.492593 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f40043-f41e-4642-8987-75a3bc95ec6e service glance] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:29.496054 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:16:29.503700 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1151/2296] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:16:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:16:39.404929 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=51,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:16:39.506345 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:16:47.892560 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597bf22-5d9b-4fa0-b8bc-0efe3a607749 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:47.894126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-d597bf22-5d9b-4fa0-b8bc-0efe3a607749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:47.955686 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597bf22-5d9b-4fa0-b8bc-0efe3a607749 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:47.959193 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc9baa-379a-4e99-b937-e0e72ea2e778 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:47.959787 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc9baa-379a-4e99-b937-e0e72ea2e778 service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:47.960062 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc9baa-379a-4e99-b937-e0e72ea2e778 service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:47.960328 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc9baa-379a-4e99-b937-e0e72ea2e778 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:47.961997 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-2ecc9baa-379a-4e99-b937-e0e72ea2e778 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:48.024037 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecc9baa-379a-4e99-b937-e0e72ea2e778 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:48.025453 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecc9baa-379a-4e99-b937-e0e72ea2e778 service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:48.035162 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1146/2297] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:16:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:16:51.310465 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f40043-f41e-4642-8987-75a3bc95ec6e service glance] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:51.312242 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a3f40043-f41e-4642-8987-75a3bc95ec6e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:51.326795 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f40043-f41e-4642-8987-75a3bc95ec6e service glance] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:51.331408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b4eb4-fc93-47e9-9eb5-689fdf0ca21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:51.332034 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b4eb4-fc93-47e9-9eb5-689fdf0ca21b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:51.332309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b4eb4-fc93-47e9-9eb5-689fdf0ca21b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:51.332577 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b4eb4-fc93-47e9-9eb5-689fdf0ca21b admin admin] PATH_INFO: `/v3/users/b5dbaf53464b44fe8d9e16f5013587b4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:51.333707 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b4eb4-fc93-47e9-9eb5-689fdf0ca21b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5dbaf53464b44fe8d9e16f5013587b4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:51.335642 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b4eb4-fc93-47e9-9eb5-689fdf0ca21b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:51.389106 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-873b4eb4-fc93-47e9-9eb5-689fdf0ca21b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5dbaf53464b44fe8d9e16f5013587b4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:51.396274 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-873b4eb4-fc93-47e9-9eb5-689fdf0ca21b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5dbaf53464b44fe8d9e16f5013587b4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:51.396939 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-873b4eb4-fc93-47e9-9eb5-689fdf0ca21b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5dbaf53464b44fe8d9e16f5013587b4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:51.404727 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-873b4eb4-fc93-47e9-9eb5-689fdf0ca21b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5dbaf53464b44fe8d9e16f5013587b4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:51.416599 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-873b4eb4-fc93-47e9-9eb5-689fdf0ca21b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5dbaf53464b44fe8d9e16f5013587b4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:51.422942 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1152/2298] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:16:51 2022] DELETE /identity/v3/users/b5dbaf53464b44fe8d9e16f5013587b4 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:16:51.604430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc9baa-379a-4e99-b937-e0e72ea2e778 service neutron] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:51.616854 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc9baa-379a-4e99-b937-e0e72ea2e778 service neutron] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:51.673253 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab92cf-88cc-48b2-9a7b-a1530f4e1679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:51.674125 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab92cf-88cc-48b2-9a7b-a1530f4e1679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:51.674400 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab92cf-88cc-48b2-9a7b-a1530f4e1679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:51.674673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab92cf-88cc-48b2-9a7b-a1530f4e1679 admin admin] PATH_INFO: `/v3/projects/4af4b9833fd14a1f881610bbc90bbd37` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:51.675816 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab92cf-88cc-48b2-9a7b-a1530f4e1679 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4af4b9833fd14a1f881610bbc90bbd37)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:51.677693 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab92cf-88cc-48b2-9a7b-a1530f4e1679 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:51.722914 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c0ab92cf-88cc-48b2-9a7b-a1530f4e1679 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4af4b9833fd14a1f881610bbc90bbd37', 'request_id': 'req-c0ab92cf-88cc-48b2-9a7b-a1530f4e1679'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:51.724846 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:16:51.728989 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c0ab92cf-88cc-48b2-9a7b-a1530f4e1679 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4af4b9833fd14a1f881610bbc90bbd37', 'request_id': 'req-c0ab92cf-88cc-48b2-9a7b-a1530f4e1679'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:51.736676 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c0ab92cf-88cc-48b2-9a7b-a1530f4e1679 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4af4b9833fd14a1f881610bbc90bbd37', 'request_id': 'req-c0ab92cf-88cc-48b2-9a7b-a1530f4e1679'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:51.748244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c0ab92cf-88cc-48b2-9a7b-a1530f4e1679 admin admin] The token cache is being invalidate because project 4af4b9833fd14a1f881610bbc90bbd37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:16:51.748526 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c0ab92cf-88cc-48b2-9a7b-a1530f4e1679 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:51.749133 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c0ab92cf-88cc-48b2-9a7b-a1530f4e1679 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:51.750586 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1147/2299] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:16:51 2022] DELETE /identity/v3/projects/4af4b9833fd14a1f881610bbc90bbd37 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:16:51.755735 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eae0bf-2b85-4c36-9189-8f62c49704ba None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:51.755952 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eae0bf-2b85-4c36-9189-8f62c49704ba None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:51.756219 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eae0bf-2b85-4c36-9189-8f62c49704ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:51.775638 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-59eae0bf-2b85-4c36-9189-8f62c49704ba None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:16:51.776812 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-59eae0bf-2b85-4c36-9189-8f62c49704ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:51.833597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1153/2300] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:16:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:16:51.841747 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cf892-e7de-4c8b-ab98-9fd40d6415af None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:51.842032 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cf892-e7de-4c8b-ab98-9fd40d6415af None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:51.842306 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cf892-e7de-4c8b-ab98-9fd40d6415af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:51.861851 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-612cf892-e7de-4c8b-ab98-9fd40d6415af None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:16:51.863107 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-612cf892-e7de-4c8b-ab98-9fd40d6415af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:51.870881 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1148/2301] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:16:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:16:51.874909 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eae0bf-2b85-4c36-9189-8f62c49704ba None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:51.884295 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eae0bf-2b85-4c36-9189-8f62c49704ba None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:51.889263 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb12ab-a873-417c-b826-ac46f36530e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:51.889855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb12ab-a873-417c-b826-ac46f36530e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:51.890130 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb12ab-a873-417c-b826-ac46f36530e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:51.890396 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fb12ab-a873-417c-b826-ac46f36530e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:51.891034 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fb12ab-a873-417c-b826-ac46f36530e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:16:51.891358 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fb12ab-a873-417c-b826-ac46f36530e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:51.892926 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fb12ab-a873-417c-b826-ac46f36530e9 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:51.904607 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1154/2302] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:16:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:16:51.908823 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612cf892-e7de-4c8b-ab98-9fd40d6415af None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:51.917912 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612cf892-e7de-4c8b-ab98-9fd40d6415af None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:51.922563 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f5846-52f6-469f-8c21-7627672adfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:51.923186 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f5846-52f6-469f-8c21-7627672adfba admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:51.923723 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f5846-52f6-469f-8c21-7627672adfba admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:51.923986 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f5846-52f6-469f-8c21-7627672adfba admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:51.924622 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f5846-52f6-469f-8c21-7627672adfba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:51.926500 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f5846-52f6-469f-8c21-7627672adfba admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:51.952973 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1149/2303] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:16:51 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:16:51.957197 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb12ab-a873-417c-b826-ac46f36530e9 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:51.967401 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fb12ab-a873-417c-b826-ac46f36530e9 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:52.023139 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096467fa-211b-45a5-b66a-122184dbdfaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:52.023677 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096467fa-211b-45a5-b66a-122184dbdfaa admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:52.023955 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096467fa-211b-45a5-b66a-122184dbdfaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:52.024226 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096467fa-211b-45a5-b66a-122184dbdfaa admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:52.024856 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096467fa-211b-45a5-b66a-122184dbdfaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:52.026750 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096467fa-211b-45a5-b66a-122184dbdfaa admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:52.035185 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:16:52.049974 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1155/2304] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:16:51 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:16:52.054018 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f5846-52f6-469f-8c21-7627672adfba admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:52.064697 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f5846-52f6-469f-8c21-7627672adfba admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:52.069264 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc5fc7-001e-44a5-a6a6-91695adc4785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:52.069985 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc5fc7-001e-44a5-a6a6-91695adc4785 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:52.070285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc5fc7-001e-44a5-a6a6-91695adc4785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:52.070558 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc5fc7-001e-44a5-a6a6-91695adc4785 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:52.071235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bc5fc7-001e-44a5-a6a6-91695adc4785 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:52.073096 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bc5fc7-001e-44a5-a6a6-91695adc4785 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:52.083624 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1150/2305] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:16:52.087634 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096467fa-211b-45a5-b66a-122184dbdfaa admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:52.098517 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096467fa-211b-45a5-b66a-122184dbdfaa admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:52.103241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba33d872-3b32-4204-87d3-b9b899d6bc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:52.103835 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33d872-3b32-4204-87d3-b9b899d6bc8b admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:52.104135 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33d872-3b32-4204-87d3-b9b899d6bc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:52.104403 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33d872-3b32-4204-87d3-b9b899d6bc8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:52.105036 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba33d872-3b32-4204-87d3-b9b899d6bc8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:52.106617 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba33d872-3b32-4204-87d3-b9b899d6bc8b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:52.117073 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1156/2306] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:16:52.121293 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc5fc7-001e-44a5-a6a6-91695adc4785 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:52.131705 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc5fc7-001e-44a5-a6a6-91695adc4785 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:52.136244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb0a01-050b-458f-9e1e-69ebbedcbdbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:52.136914 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb0a01-050b-458f-9e1e-69ebbedcbdbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:52.137741 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb0a01-050b-458f-9e1e-69ebbedcbdbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:52.138028 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb0a01-050b-458f-9e1e-69ebbedcbdbc admin admin] PATH_INFO: `/v3/projects/da546353194342439f9bc3f98f6737e0/users/2cd12070eb954d6880f925957184a288/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:52.152705 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cb0a01-050b-458f-9e1e-69ebbedcbdbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=da546353194342439f9bc3f98f6737e0, user_id=2cd12070eb954d6880f925957184a288, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:52.155377 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cb0a01-050b-458f-9e1e-69ebbedcbdbc admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:52.168174 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1151/2307] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:16:52 2022] PUT /identity/v3/projects/da546353194342439f9bc3f98f6737e0/users/2cd12070eb954d6880f925957184a288/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:16:52.173086 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd8e95-fd92-4c5b-a137-cf91e2a39f55 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:52.173628 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd8e95-fd92-4c5b-a137-cf91e2a39f55 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:52.173913 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd8e95-fd92-4c5b-a137-cf91e2a39f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:52.192664 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f4fd8e95-fd92-4c5b-a137-cf91e2a39f55 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:16:52.193859 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f4fd8e95-fd92-4c5b-a137-cf91e2a39f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:52.252918 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1157/2308] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:16:52.263039 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb0a01-050b-458f-9e1e-69ebbedcbdbc admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:52.264750 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-b2cb0a01-050b-458f-9e1e-69ebbedcbdbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:52.319436 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb0a01-050b-458f-9e1e-69ebbedcbdbc admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:52.329080 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1fa0f-633c-40dd-adae-4e815df97e7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:52.329699 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1fa0f-633c-40dd-adae-4e815df97e7d service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:52.329980 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1fa0f-633c-40dd-adae-4e815df97e7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:52.330249 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1fa0f-633c-40dd-adae-4e815df97e7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:52.339794 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb1fa0f-633c-40dd-adae-4e815df97e7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:52.341221 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb1fa0f-633c-40dd-adae-4e815df97e7d service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:52.350502 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1152/2309] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:16:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:16:54.334359 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd8e95-fd92-4c5b-a137-cf91e2a39f55 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:54.335848 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f4fd8e95-fd92-4c5b-a137-cf91e2a39f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:54.348184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd8e95-fd92-4c5b-a137-cf91e2a39f55 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:54.353100 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0f04a-86cc-4ff0-b7bf-749cddaad78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:54.353735 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0f04a-86cc-4ff0-b7bf-749cddaad78a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:54.354009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0f04a-86cc-4ff0-b7bf-749cddaad78a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:54.354276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0f04a-86cc-4ff0-b7bf-749cddaad78a admin admin] PATH_INFO: `/v3/users/2ae75f5b6c70464696d5ecba26bbc8e7` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:54.355449 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf0f04a-86cc-4ff0-b7bf-749cddaad78a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ae75f5b6c70464696d5ecba26bbc8e7)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:54.357555 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf0f04a-86cc-4ff0-b7bf-749cddaad78a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:54.425220 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6bf0f04a-86cc-4ff0-b7bf-749cddaad78a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ae75f5b6c70464696d5ecba26bbc8e7'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:54.436611 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6bf0f04a-86cc-4ff0-b7bf-749cddaad78a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ae75f5b6c70464696d5ecba26bbc8e7'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:54.437336 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6bf0f04a-86cc-4ff0-b7bf-749cddaad78a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ae75f5b6c70464696d5ecba26bbc8e7'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:54.447220 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6bf0f04a-86cc-4ff0-b7bf-749cddaad78a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ae75f5b6c70464696d5ecba26bbc8e7'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:54.463404 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6bf0f04a-86cc-4ff0-b7bf-749cddaad78a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ae75f5b6c70464696d5ecba26bbc8e7'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:54.472133 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1158/2310] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:16:54 2022] DELETE /identity/v3/users/2ae75f5b6c70464696d5ecba26bbc8e7 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:16:54.476478 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e07efd-1c87-4d1d-9cb2-df5a4ccbff95 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:54.476959 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e07efd-1c87-4d1d-9cb2-df5a4ccbff95 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:54.477081 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e07efd-1c87-4d1d-9cb2-df5a4ccbff95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:54.503412 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-03e07efd-1c87-4d1d-9cb2-df5a4ccbff95 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:16:54.506037 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-03e07efd-1c87-4d1d-9cb2-df5a4ccbff95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:54.576730 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1153/2311] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:16:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:16:54.587636 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0f04a-86cc-4ff0-b7bf-749cddaad78a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:54.589319 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6bf0f04a-86cc-4ff0-b7bf-749cddaad78a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:54.672083 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0f04a-86cc-4ff0-b7bf-749cddaad78a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:54.676440 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29371318-8e7d-4267-8449-9248daba2406 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:54.677238 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29371318-8e7d-4267-8449-9248daba2406 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:54.677525 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29371318-8e7d-4267-8449-9248daba2406 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:54.677800 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29371318-8e7d-4267-8449-9248daba2406 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:54.685141 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:16:54.690858 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29371318-8e7d-4267-8449-9248daba2406 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:54.692795 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29371318-8e7d-4267-8449-9248daba2406 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:54.723652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1159/2312] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:16:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:16:55.098080 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e07efd-1c87-4d1d-9cb2-df5a4ccbff95 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:55.100554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-03e07efd-1c87-4d1d-9cb2-df5a4ccbff95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:55.139879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e07efd-1c87-4d1d-9cb2-df5a4ccbff95 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:55.179259 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2920a1e-983c-45d5-bbc4-187f51054917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:55.180201 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2920a1e-983c-45d5-bbc4-187f51054917 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:55.180639 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2920a1e-983c-45d5-bbc4-187f51054917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:55.181067 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2920a1e-983c-45d5-bbc4-187f51054917 admin admin] PATH_INFO: `/v3/projects/3a5a12da1f8944bb9e447fb28ee20e66` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:55.185474 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2920a1e-983c-45d5-bbc4-187f51054917 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a5a12da1f8944bb9e447fb28ee20e66)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:55.195501 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2920a1e-983c-45d5-bbc4-187f51054917 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:55.251129 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:16:55.310778 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-d2920a1e-983c-45d5-bbc4-187f51054917 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a5a12da1f8944bb9e447fb28ee20e66', 'request_id': 'req-d2920a1e-983c-45d5-bbc4-187f51054917'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:55.321675 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-d2920a1e-983c-45d5-bbc4-187f51054917 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a5a12da1f8944bb9e447fb28ee20e66', 'request_id': 'req-d2920a1e-983c-45d5-bbc4-187f51054917'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:55.331574 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-d2920a1e-983c-45d5-bbc4-187f51054917 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a5a12da1f8944bb9e447fb28ee20e66', 'request_id': 'req-d2920a1e-983c-45d5-bbc4-187f51054917'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:55.351664 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-d2920a1e-983c-45d5-bbc4-187f51054917 admin admin] The token cache is being invalidate because project 3a5a12da1f8944bb9e447fb28ee20e66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:16:55.351971 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-d2920a1e-983c-45d5-bbc4-187f51054917 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:55.352648 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-d2920a1e-983c-45d5-bbc4-187f51054917 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:55.357761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1154/2313] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:16:55 2022] DELETE /identity/v3/projects/3a5a12da1f8944bb9e447fb28ee20e66 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:16:55.359137 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29371318-8e7d-4267-8449-9248daba2406 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:55.370698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29371318-8e7d-4267-8449-9248daba2406 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:55.438563 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1612a680-2741-4118-8bbb-20ff8e7e52b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:55.439286 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612a680-2741-4118-8bbb-20ff8e7e52b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:55.439562 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612a680-2741-4118-8bbb-20ff8e7e52b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:55.439839 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612a680-2741-4118-8bbb-20ff8e7e52b8 admin admin] PATH_INFO: `/v3/users/24f63cf37354457e8f6b689795de418d` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:55.441017 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1612a680-2741-4118-8bbb-20ff8e7e52b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24f63cf37354457e8f6b689795de418d)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:55.443042 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1612a680-2741-4118-8bbb-20ff8e7e52b8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:55.507991 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1612a680-2741-4118-8bbb-20ff8e7e52b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24f63cf37354457e8f6b689795de418d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:55.517060 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1612a680-2741-4118-8bbb-20ff8e7e52b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24f63cf37354457e8f6b689795de418d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:55.517920 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1612a680-2741-4118-8bbb-20ff8e7e52b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24f63cf37354457e8f6b689795de418d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:55.527835 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1612a680-2741-4118-8bbb-20ff8e7e52b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24f63cf37354457e8f6b689795de418d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:55.545177 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1612a680-2741-4118-8bbb-20ff8e7e52b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24f63cf37354457e8f6b689795de418d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:55.553886 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1160/2314] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:16:55 2022] DELETE /identity/v3/users/24f63cf37354457e8f6b689795de418d => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:16:55.922822 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2920a1e-983c-45d5-bbc4-187f51054917 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:55.928153 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-d2920a1e-983c-45d5-bbc4-187f51054917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:55.948959 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2920a1e-983c-45d5-bbc4-187f51054917 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:56.028064 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3159bda-36b4-410a-abac-3831a999eb7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:56.028558 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3159bda-36b4-410a-abac-3831a999eb7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:56.028855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3159bda-36b4-410a-abac-3831a999eb7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:56.029101 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3159bda-36b4-410a-abac-3831a999eb7d admin admin] PATH_INFO: `/v3/projects/4d65446ad1ff462ebbddb400362dfc1d` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:56.030364 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3159bda-36b4-410a-abac-3831a999eb7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d65446ad1ff462ebbddb400362dfc1d)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:56.032324 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3159bda-36b4-410a-abac-3831a999eb7d admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:56.085663 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-a3159bda-36b4-410a-abac-3831a999eb7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d65446ad1ff462ebbddb400362dfc1d', 'request_id': 'req-a3159bda-36b4-410a-abac-3831a999eb7d'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:56.091790 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-a3159bda-36b4-410a-abac-3831a999eb7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d65446ad1ff462ebbddb400362dfc1d', 'request_id': 'req-a3159bda-36b4-410a-abac-3831a999eb7d'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:56.098207 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-a3159bda-36b4-410a-abac-3831a999eb7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d65446ad1ff462ebbddb400362dfc1d', 'request_id': 'req-a3159bda-36b4-410a-abac-3831a999eb7d'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:56.114571 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-a3159bda-36b4-410a-abac-3831a999eb7d admin admin] The token cache is being invalidate because project 4d65446ad1ff462ebbddb400362dfc1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:16:56.114571 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-a3159bda-36b4-410a-abac-3831a999eb7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:56.116002 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-a3159bda-36b4-410a-abac-3831a999eb7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:16:56.117011 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1155/2315] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:16:55 2022] DELETE /identity/v3/projects/4d65446ad1ff462ebbddb400362dfc1d => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:16:56.122194 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8898e5-0202-4a10-a478-58ec2e4aa5a5 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:56.122491 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8898e5-0202-4a10-a478-58ec2e4aa5a5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:56.123162 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8898e5-0202-4a10-a478-58ec2e4aa5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:56.148115 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-9e8898e5-0202-4a10-a478-58ec2e4aa5a5 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:16:56.149460 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9e8898e5-0202-4a10-a478-58ec2e4aa5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:56.221769 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1161/2316] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:16:56.231399 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a09754e-101d-4ff4-a66c-c928abd2d636 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:56.231706 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a09754e-101d-4ff4-a66c-c928abd2d636 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:56.232023 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a09754e-101d-4ff4-a66c-c928abd2d636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:56.257658 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-0a09754e-101d-4ff4-a66c-c928abd2d636 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:16:56.258868 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-0a09754e-101d-4ff4-a66c-c928abd2d636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:56.267031 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1156/2317] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:16:56.272720 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8898e5-0202-4a10-a478-58ec2e4aa5a5 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:56.284249 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8898e5-0202-4a10-a478-58ec2e4aa5a5 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:56.291006 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb61c4-e799-4b3e-9adb-7921f711e562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:56.291370 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb61c4-e799-4b3e-9adb-7921f711e562 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:56.292084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb61c4-e799-4b3e-9adb-7921f711e562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:56.292084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb61c4-e799-4b3e-9adb-7921f711e562 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:56.293029 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cb61c4-e799-4b3e-9adb-7921f711e562 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:16:56.293877 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cb61c4-e799-4b3e-9adb-7921f711e562 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:56.295616 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cb61c4-e799-4b3e-9adb-7921f711e562 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:56.306043 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:16:56.315260 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1162/2318] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:16:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:16:56.315469 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a09754e-101d-4ff4-a66c-c928abd2d636 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:56.327945 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a09754e-101d-4ff4-a66c-c928abd2d636 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:56.335930 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b414b5f-694c-4837-9400-4512dde0fec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:56.337423 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b414b5f-694c-4837-9400-4512dde0fec0 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:56.338018 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b414b5f-694c-4837-9400-4512dde0fec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:56.338981 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b414b5f-694c-4837-9400-4512dde0fec0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:56.339926 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b414b5f-694c-4837-9400-4512dde0fec0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:56.342434 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b414b5f-694c-4837-9400-4512dde0fec0 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:56.419995 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1157/2319] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:16:56 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:16:56.421665 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb61c4-e799-4b3e-9adb-7921f711e562 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:56.437796 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb61c4-e799-4b3e-9adb-7921f711e562 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:56.543150 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049fbed-e4dd-47f8-8875-b66b9907a6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:56.543880 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049fbed-e4dd-47f8-8875-b66b9907a6e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:56.544467 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049fbed-e4dd-47f8-8875-b66b9907a6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:56.545483 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049fbed-e4dd-47f8-8875-b66b9907a6e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:56.546376 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049fbed-e4dd-47f8-8875-b66b9907a6e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:56.548978 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049fbed-e4dd-47f8-8875-b66b9907a6e5 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:56.584701 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1163/2320] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:16:56 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:16:56.588644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b414b5f-694c-4837-9400-4512dde0fec0 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:56.596434 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:16:56.605987 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b414b5f-694c-4837-9400-4512dde0fec0 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:56.619882 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9acb6a9-494a-41b8-ad59-fb8840c12912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:56.620559 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acb6a9-494a-41b8-ad59-fb8840c12912 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:56.620838 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acb6a9-494a-41b8-ad59-fb8840c12912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:56.621108 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acb6a9-494a-41b8-ad59-fb8840c12912 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:56.624658 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9acb6a9-494a-41b8-ad59-fb8840c12912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:56.628381 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9acb6a9-494a-41b8-ad59-fb8840c12912 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:56.653396 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1158/2321] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:16:56.657702 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049fbed-e4dd-47f8-8875-b66b9907a6e5 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:56.670849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39267136-d487-44e5-89a8-aba577b54013 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:56.671178 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39267136-d487-44e5-89a8-aba577b54013 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:56.671432 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39267136-d487-44e5-89a8-aba577b54013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:56.680374 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049fbed-e4dd-47f8-8875-b66b9907a6e5 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:56.687293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71fd948-6ba0-4ef7-8a71-e0d8fe1d2df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:56.688404 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71fd948-6ba0-4ef7-8a71-e0d8fe1d2df3 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:56.689010 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71fd948-6ba0-4ef7-8a71-e0d8fe1d2df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:56.689350 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71fd948-6ba0-4ef7-8a71-e0d8fe1d2df3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:56.690058 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71fd948-6ba0-4ef7-8a71-e0d8fe1d2df3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:56.691884 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71fd948-6ba0-4ef7-8a71-e0d8fe1d2df3 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:56.705562 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1164/2322] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:16:56.710369 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71fd948-6ba0-4ef7-8a71-e0d8fe1d2df3 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:56.711373 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-39267136-d487-44e5-89a8-aba577b54013 None None] MFA Rules not processed for user `2cd12070eb954d6880f925957184a288`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:16:56.712833 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-39267136-d487-44e5-89a8-aba577b54013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:56.720738 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71fd948-6ba0-4ef7-8a71-e0d8fe1d2df3 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:56.726485 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf7f91c-6195-4ff1-b1cd-e97a9beeba5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:56.726896 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7f91c-6195-4ff1-b1cd-e97a9beeba5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:56.727225 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7f91c-6195-4ff1-b1cd-e97a9beeba5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:56.727501 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7f91c-6195-4ff1-b1cd-e97a9beeba5c admin admin] PATH_INFO: `/v3/projects/b5b9591bcb6844089750ccac49bdc5c6/users/aadfec5f9c22431caf0fb9c306baafb7/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:56.747300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf7f91c-6195-4ff1-b1cd-e97a9beeba5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5b9591bcb6844089750ccac49bdc5c6, user_id=aadfec5f9c22431caf0fb9c306baafb7, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:56.750454 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf7f91c-6195-4ff1-b1cd-e97a9beeba5c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:56.791346 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1165/2323] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:16:56 2022] PUT /identity/v3/projects/b5b9591bcb6844089750ccac49bdc5c6/users/aadfec5f9c22431caf0fb9c306baafb7/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:16:56.804219 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cac8f1-8c2e-47d0-999d-ad9cedb78abe None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:56.804219 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cac8f1-8c2e-47d0-999d-ad9cedb78abe None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:56.804219 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cac8f1-8c2e-47d0-999d-ad9cedb78abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:56.851082 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1159/2324] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:16:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:16:56.854953 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39267136-d487-44e5-89a8-aba577b54013 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:56.856859 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-39267136-d487-44e5-89a8-aba577b54013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:56.872957 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-a0cac8f1-8c2e-47d0-999d-ad9cedb78abe None None] MFA Rules not processed for user `aadfec5f9c22431caf0fb9c306baafb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:16:56.874673 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a0cac8f1-8c2e-47d0-999d-ad9cedb78abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:56.954233 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39267136-d487-44e5-89a8-aba577b54013 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:56.969696 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1166/2325] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:16:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:16:56.974047 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cac8f1-8c2e-47d0-999d-ad9cedb78abe None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:57.035907 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f743b-c474-45e0-9c99-0afd05a865f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:57.035907 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f743b-c474-45e0-9c99-0afd05a865f6 service nova] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:57.035907 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f743b-c474-45e0-9c99-0afd05a865f6 service nova] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:57.035907 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f743b-c474-45e0-9c99-0afd05a865f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:57.047866 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:16:57.079096 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:16:57.082045 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cac8f1-8c2e-47d0-999d-ad9cedb78abe None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:57.089620 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15df88b-0e91-4683-b423-849570c1880b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:57.090374 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15df88b-0e91-4683-b423-849570c1880b admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:57.090719 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15df88b-0e91-4683-b423-849570c1880b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:57.091077 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15df88b-0e91-4683-b423-849570c1880b admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:57.091815 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15df88b-0e91-4683-b423-849570c1880b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:57.095110 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15df88b-0e91-4683-b423-849570c1880b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:57.128019 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833f743b-c474-45e0-9c99-0afd05a865f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:57.129846 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833f743b-c474-45e0-9c99-0afd05a865f6 service nova] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:57.137409 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1167/2326] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:16:56 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:16:57.141524 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15df88b-0e91-4683-b423-849570c1880b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:57.158319 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1160/2327] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:16:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:16:57.239284 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15df88b-0e91-4683-b423-849570c1880b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:57.245153 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9d6f0-726c-435f-9f8f-c5e175baea09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:57.246293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9d6f0-726c-435f-9f8f-c5e175baea09 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:57.247463 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9d6f0-726c-435f-9f8f-c5e175baea09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:57.247814 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9d6f0-726c-435f-9f8f-c5e175baea09 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:57.248747 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9d6f0-726c-435f-9f8f-c5e175baea09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:57.251003 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9d6f0-726c-435f-9f8f-c5e175baea09 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:57.287285 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1168/2328] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:16:57 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:16:57.288630 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f743b-c474-45e0-9c99-0afd05a865f6 service nova] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:57.308493 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f743b-c474-45e0-9c99-0afd05a865f6 service nova] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:57.317062 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd4ccea-7148-48bd-9c9c-61bb1f5faf2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:57.318274 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4ccea-7148-48bd-9c9c-61bb1f5faf2c admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:57.318567 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4ccea-7148-48bd-9c9c-61bb1f5faf2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:57.318848 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4ccea-7148-48bd-9c9c-61bb1f5faf2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:57.319794 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd4ccea-7148-48bd-9c9c-61bb1f5faf2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:57.323126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd4ccea-7148-48bd-9c9c-61bb1f5faf2c admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:57.335027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1161/2329] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:16:57.338568 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9d6f0-726c-435f-9f8f-c5e175baea09 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:57.349112 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9d6f0-726c-435f-9f8f-c5e175baea09 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:57.354623 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9682d4d-9430-462f-b847-b1579cd66fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:57.355297 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9682d4d-9430-462f-b847-b1579cd66fa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:57.355577 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9682d4d-9430-462f-b847-b1579cd66fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:57.356470 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9682d4d-9430-462f-b847-b1579cd66fa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:57.357037 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9682d4d-9430-462f-b847-b1579cd66fa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:57.358667 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9682d4d-9430-462f-b847-b1579cd66fa9 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:57.369618 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1169/2330] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:16:57.375007 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd4ccea-7148-48bd-9c9c-61bb1f5faf2c admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:57.387368 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd4ccea-7148-48bd-9c9c-61bb1f5faf2c admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:57.400486 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f006f-eff2-43d7-9094-eeeea22e081c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:57.401170 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f006f-eff2-43d7-9094-eeeea22e081c admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:57.401448 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f006f-eff2-43d7-9094-eeeea22e081c admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:57.401722 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80f006f-eff2-43d7-9094-eeeea22e081c admin admin] PATH_INFO: `/v3/projects/37031c4080a14e45aa7531e20394c6fb/users/2978b4738979401cb3f70128c38ef826/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:57.419271 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f006f-eff2-43d7-9094-eeeea22e081c admin admin] RBAC: Authorizing `identity:create_grant(project_id=37031c4080a14e45aa7531e20394c6fb, user_id=2978b4738979401cb3f70128c38ef826, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:57.422254 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80f006f-eff2-43d7-9094-eeeea22e081c admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:57.444079 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1162/2331] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:16:57 2022] PUT /identity/v3/projects/37031c4080a14e45aa7531e20394c6fb/users/2978b4738979401cb3f70128c38ef826/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:16:57.454917 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81224e51-6f86-44b2-b6d1-25b7d498c1b2 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:57.455503 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81224e51-6f86-44b2-b6d1-25b7d498c1b2 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:57.455742 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81224e51-6f86-44b2-b6d1-25b7d498c1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:57.489864 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-81224e51-6f86-44b2-b6d1-25b7d498c1b2 None None] MFA Rules not processed for user `2978b4738979401cb3f70128c38ef826`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:16:57.491134 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-81224e51-6f86-44b2-b6d1-25b7d498c1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:57.576022 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1170/2332] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:16:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:16:57.581055 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f006f-eff2-43d7-9094-eeeea22e081c admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:57.679745 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80f006f-eff2-43d7-9094-eeeea22e081c admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:57.684655 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebcb4c0-3bf0-463d-9205-2b4c1f82452b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:57.685410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcb4c0-3bf0-463d-9205-2b4c1f82452b admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:57.685703 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcb4c0-3bf0-463d-9205-2b4c1f82452b admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:57.685976 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcb4c0-3bf0-463d-9205-2b4c1f82452b admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:57.686621 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebcb4c0-3bf0-463d-9205-2b4c1f82452b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:57.688632 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebcb4c0-3bf0-463d-9205-2b4c1f82452b admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:57.724957 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1163/2333] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:16:57 2022] POST /identity/v3/projects => generated 377 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:16:57.729036 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81224e51-6f86-44b2-b6d1-25b7d498c1b2 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:57.745106 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81224e51-6f86-44b2-b6d1-25b7d498c1b2 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:57.784223 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:16:57.814105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be97dbe-0f7a-493c-8bdd-25eb240b4485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:57.814728 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be97dbe-0f7a-493c-8bdd-25eb240b4485 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:57.815721 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be97dbe-0f7a-493c-8bdd-25eb240b4485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:57.816002 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be97dbe-0f7a-493c-8bdd-25eb240b4485 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:57.816721 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be97dbe-0f7a-493c-8bdd-25eb240b4485 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:57.818888 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be97dbe-0f7a-493c-8bdd-25eb240b4485 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:57.849575 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1171/2334] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:16:57 2022] POST /identity/v3/users => generated 295 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:16:57.850248 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebcb4c0-3bf0-463d-9205-2b4c1f82452b admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:57.862643 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebcb4c0-3bf0-463d-9205-2b4c1f82452b admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:57.867897 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e38a41-cc53-4514-9363-70b6cf69894e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:57.868512 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e38a41-cc53-4514-9363-70b6cf69894e admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:57.868784 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e38a41-cc53-4514-9363-70b6cf69894e admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:57.869051 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e38a41-cc53-4514-9363-70b6cf69894e admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:57.869694 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e38a41-cc53-4514-9363-70b6cf69894e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:57.871357 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e38a41-cc53-4514-9363-70b6cf69894e admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:57.884245 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1164/2335] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:16:57.888763 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be97dbe-0f7a-493c-8bdd-25eb240b4485 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:57.899869 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be97dbe-0f7a-493c-8bdd-25eb240b4485 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:57.911431 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56bbec6-d3e3-49af-973f-4dbeb80d9bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:57.912195 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56bbec6-d3e3-49af-973f-4dbeb80d9bce admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:57.912476 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56bbec6-d3e3-49af-973f-4dbeb80d9bce admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:57.912750 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56bbec6-d3e3-49af-973f-4dbeb80d9bce admin admin] PATH_INFO: `/v3/projects/77635c4d233345edb8736b4fbe3d8cb8/users/b0758314c4ab4cfd97f31d72e947fe52/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:57.941199 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56bbec6-d3e3-49af-973f-4dbeb80d9bce admin admin] RBAC: Authorizing `identity:create_grant(project_id=77635c4d233345edb8736b4fbe3d8cb8, user_id=b0758314c4ab4cfd97f31d72e947fe52, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:57.943970 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56bbec6-d3e3-49af-973f-4dbeb80d9bce admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:57.960399 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1172/2336] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:16:57 2022] PUT /identity/v3/projects/77635c4d233345edb8736b4fbe3d8cb8/users/b0758314c4ab4cfd97f31d72e947fe52/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:16:57.964540 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e38a41-cc53-4514-9363-70b6cf69894e admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:57.977679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e38a41-cc53-4514-9363-70b6cf69894e admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:58.037220 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:16:58.044638 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450c24d3-749b-4219-9d4b-97005c0551bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:58.045806 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c24d3-749b-4219-9d4b-97005c0551bc admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:58.046192 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c24d3-749b-4219-9d4b-97005c0551bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:58.047556 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c24d3-749b-4219-9d4b-97005c0551bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:58.049537 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450c24d3-749b-4219-9d4b-97005c0551bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:58.052241 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450c24d3-749b-4219-9d4b-97005c0551bc admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:58.066127 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1165/2337] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:16:58.070246 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56bbec6-d3e3-49af-973f-4dbeb80d9bce admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:58.081776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56bbec6-d3e3-49af-973f-4dbeb80d9bce admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:58.087213 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebcd159-63ed-4af2-9cbe-e3e8d9bee8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:58.087892 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebcd159-63ed-4af2-9cbe-e3e8d9bee8a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:58.088169 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebcd159-63ed-4af2-9cbe-e3e8d9bee8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:58.088446 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebcd159-63ed-4af2-9cbe-e3e8d9bee8a3 admin admin] PATH_INFO: `/v3/projects/77635c4d233345edb8736b4fbe3d8cb8/users/b0758314c4ab4cfd97f31d72e947fe52/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:58.091030 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebcd159-63ed-4af2-9cbe-e3e8d9bee8a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77635c4d233345edb8736b4fbe3d8cb8, user_id=b0758314c4ab4cfd97f31d72e947fe52, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:58.093748 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebcd159-63ed-4af2-9cbe-e3e8d9bee8a3 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:58.112156 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1173/2338] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:16:58 2022] PUT /identity/v3/projects/77635c4d233345edb8736b4fbe3d8cb8/users/b0758314c4ab4cfd97f31d72e947fe52/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:16:58.123766 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9167c83-9cc3-407f-9a78-767797bcd2da None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:58.124053 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9167c83-9cc3-407f-9a78-767797bcd2da None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:58.124332 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9167c83-9cc3-407f-9a78-767797bcd2da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:58.156389 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-f9167c83-9cc3-407f-9a78-767797bcd2da None None] MFA Rules not processed for user `b0758314c4ab4cfd97f31d72e947fe52`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:16:58.157770 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-f9167c83-9cc3-407f-9a78-767797bcd2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:58.243698 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1166/2339] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:16:58 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:16:58.255852 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebcd159-63ed-4af2-9cbe-e3e8d9bee8a3 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:58.257615 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0ebcd159-63ed-4af2-9cbe-e3e8d9bee8a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:58.329323 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebcd159-63ed-4af2-9cbe-e3e8d9bee8a3 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:58.333272 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c62d83-2d93-469b-9d2f-ff9d763264a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:58.333828 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c62d83-2d93-469b-9d2f-ff9d763264a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:58.334109 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c62d83-2d93-469b-9d2f-ff9d763264a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:58.334384 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c62d83-2d93-469b-9d2f-ff9d763264a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:58.343769 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c62d83-2d93-469b-9d2f-ff9d763264a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:58.345198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c62d83-2d93-469b-9d2f-ff9d763264a4 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:58.355046 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1174/2340] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:16:58 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:16:58.397567 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9167c83-9cc3-407f-9a78-767797bcd2da None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:58.399075 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-f9167c83-9cc3-407f-9a78-767797bcd2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:58.410987 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9167c83-9cc3-407f-9a78-767797bcd2da None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:58.414754 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba51ca-ffcc-4456-bfbd-2a6a0bcded7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:58.415389 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba51ca-ffcc-4456-bfbd-2a6a0bcded7a service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:58.415662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba51ca-ffcc-4456-bfbd-2a6a0bcded7a service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:58.415929 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba51ca-ffcc-4456-bfbd-2a6a0bcded7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:58.474611 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba51ca-ffcc-4456-bfbd-2a6a0bcded7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:58.476102 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba51ca-ffcc-4456-bfbd-2a6a0bcded7a service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:58.485964 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1167/2341] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:16:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:16:58.596764 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae114c2-c6b1-4fa7-a3f5-bba31904a718 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:58.597275 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae114c2-c6b1-4fa7-a3f5-bba31904a718 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:58.597965 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae114c2-c6b1-4fa7-a3f5-bba31904a718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:58.617671 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-eae114c2-c6b1-4fa7-a3f5-bba31904a718 None None] MFA Rules not processed for user `2cd12070eb954d6880f925957184a288`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:16:58.618879 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-eae114c2-c6b1-4fa7-a3f5-bba31904a718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:58.674598 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1175/2342] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:16:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:16:59.028611 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba51ca-ffcc-4456-bfbd-2a6a0bcded7a service cinder] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:16:59.030423 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-d6ba51ca-ffcc-4456-bfbd-2a6a0bcded7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:59.099309 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba51ca-ffcc-4456-bfbd-2a6a0bcded7a service cinder] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:16:59.104160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a052cf-ca7c-4832-aa91-f5bce8d4e74e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:16:59.105645 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a052cf-ca7c-4832-aa91-f5bce8d4e74e service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:16:59.105645 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a052cf-ca7c-4832-aa91-f5bce8d4e74e service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:16:59.105645 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a052cf-ca7c-4832-aa91-f5bce8d4e74e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:16:59.107956 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-94a052cf-ca7c-4832-aa91-f5bce8d4e74e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:16:59.118789 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a052cf-ca7c-4832-aa91-f5bce8d4e74e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:16:59.120987 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a052cf-ca7c-4832-aa91-f5bce8d4e74e service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:16:59.133913 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1168/2343] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:16:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:17:02.653830 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae114c2-c6b1-4fa7-a3f5-bba31904a718 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:02.729543 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:17:02.760505 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae114c2-c6b1-4fa7-a3f5-bba31904a718 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:02.766097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fcbc07-d1af-4f66-a09a-8c412ccb42df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:02.766737 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fcbc07-d1af-4f66-a09a-8c412ccb42df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:02.767017 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fcbc07-d1af-4f66-a09a-8c412ccb42df admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:02.767324 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fcbc07-d1af-4f66-a09a-8c412ccb42df admin admin] PATH_INFO: `/v3/users/aadfec5f9c22431caf0fb9c306baafb7` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:02.768488 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fcbc07-d1af-4f66-a09a-8c412ccb42df admin admin] RBAC: Authorizing `identity:delete_user(user_id=aadfec5f9c22431caf0fb9c306baafb7)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:02.770471 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fcbc07-d1af-4f66-a09a-8c412ccb42df admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:02.831781 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-37fcbc07-d1af-4f66-a09a-8c412ccb42df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aadfec5f9c22431caf0fb9c306baafb7'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:02.841287 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-37fcbc07-d1af-4f66-a09a-8c412ccb42df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aadfec5f9c22431caf0fb9c306baafb7'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:02.842122 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-37fcbc07-d1af-4f66-a09a-8c412ccb42df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aadfec5f9c22431caf0fb9c306baafb7'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:02.852364 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-37fcbc07-d1af-4f66-a09a-8c412ccb42df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aadfec5f9c22431caf0fb9c306baafb7'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:02.867663 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-37fcbc07-d1af-4f66-a09a-8c412ccb42df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aadfec5f9c22431caf0fb9c306baafb7'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:02.875807 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1176/2344] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:17:02 2022] DELETE /identity/v3/users/aadfec5f9c22431caf0fb9c306baafb7 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:17:02.880235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d66b17-6598-412c-b0b3-d5b777e98140 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:02.881090 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d66b17-6598-412c-b0b3-d5b777e98140 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:02.881536 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d66b17-6598-412c-b0b3-d5b777e98140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:02.909187 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-65d66b17-6598-412c-b0b3-d5b777e98140 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:17:02.910660 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-65d66b17-6598-412c-b0b3-d5b777e98140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:02.934695 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:17:03.007511 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1169/2345] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:17:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:17:03.018271 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fcbc07-d1af-4f66-a09a-8c412ccb42df admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:03.020515 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-37fcbc07-d1af-4f66-a09a-8c412ccb42df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:03.108162 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fcbc07-d1af-4f66-a09a-8c412ccb42df admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:03.113472 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2862e-0333-42bd-93dc-3c4bda442e21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:03.114052 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2862e-0333-42bd-93dc-3c4bda442e21 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:03.114334 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2862e-0333-42bd-93dc-3c4bda442e21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:03.114601 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2862e-0333-42bd-93dc-3c4bda442e21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:03.125488 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb2862e-0333-42bd-93dc-3c4bda442e21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:03.126956 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb2862e-0333-42bd-93dc-3c4bda442e21 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:03.137638 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1177/2346] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:17:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:17:03.455594 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d66b17-6598-412c-b0b3-d5b777e98140 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:03.457078 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-65d66b17-6598-412c-b0b3-d5b777e98140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:03.468784 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d66b17-6598-412c-b0b3-d5b777e98140 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:03.473480 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96c581-1f57-4e52-b2c4-fdd83a1dd9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:03.474102 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96c581-1f57-4e52-b2c4-fdd83a1dd9b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:03.474379 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96c581-1f57-4e52-b2c4-fdd83a1dd9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:03.474648 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96c581-1f57-4e52-b2c4-fdd83a1dd9b9 admin admin] PATH_INFO: `/v3/projects/b5b9591bcb6844089750ccac49bdc5c6` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:03.476265 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96c581-1f57-4e52-b2c4-fdd83a1dd9b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5b9591bcb6844089750ccac49bdc5c6)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:03.478297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96c581-1f57-4e52-b2c4-fdd83a1dd9b9 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:03.528525 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-ab96c581-1f57-4e52-b2c4-fdd83a1dd9b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5b9591bcb6844089750ccac49bdc5c6', 'request_id': 'req-ab96c581-1f57-4e52-b2c4-fdd83a1dd9b9'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:03.534523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-ab96c581-1f57-4e52-b2c4-fdd83a1dd9b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5b9591bcb6844089750ccac49bdc5c6', 'request_id': 'req-ab96c581-1f57-4e52-b2c4-fdd83a1dd9b9'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:03.540998 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-ab96c581-1f57-4e52-b2c4-fdd83a1dd9b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5b9591bcb6844089750ccac49bdc5c6', 'request_id': 'req-ab96c581-1f57-4e52-b2c4-fdd83a1dd9b9'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:03.554518 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-ab96c581-1f57-4e52-b2c4-fdd83a1dd9b9 admin admin] The token cache is being invalidate because project b5b9591bcb6844089750ccac49bdc5c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:17:03.554988 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-ab96c581-1f57-4e52-b2c4-fdd83a1dd9b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:03.555841 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-ab96c581-1f57-4e52-b2c4-fdd83a1dd9b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:03.557441 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1170/2347] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:17:03 2022] DELETE /identity/v3/projects/b5b9591bcb6844089750ccac49bdc5c6 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:17:03.561843 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2862e-0333-42bd-93dc-3c4bda442e21 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:03.571408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2862e-0333-42bd-93dc-3c4bda442e21 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:03.642710 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8ef15-78f3-4f6f-9dc1-ba7061a73b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:03.643456 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8ef15-78f3-4f6f-9dc1-ba7061a73b9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:03.643806 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8ef15-78f3-4f6f-9dc1-ba7061a73b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:03.644147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f8ef15-78f3-4f6f-9dc1-ba7061a73b9f admin admin] PATH_INFO: `/v3/users/2978b4738979401cb3f70128c38ef826` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:03.645349 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f8ef15-78f3-4f6f-9dc1-ba7061a73b9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2978b4738979401cb3f70128c38ef826)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:03.647412 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f8ef15-78f3-4f6f-9dc1-ba7061a73b9f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:03.648893 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:17:03.696430 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-78f8ef15-78f3-4f6f-9dc1-ba7061a73b9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2978b4738979401cb3f70128c38ef826'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:03.703705 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-78f8ef15-78f3-4f6f-9dc1-ba7061a73b9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2978b4738979401cb3f70128c38ef826'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:03.704363 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-78f8ef15-78f3-4f6f-9dc1-ba7061a73b9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2978b4738979401cb3f70128c38ef826'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:03.711740 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-78f8ef15-78f3-4f6f-9dc1-ba7061a73b9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2978b4738979401cb3f70128c38ef826'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:03.723859 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-78f8ef15-78f3-4f6f-9dc1-ba7061a73b9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2978b4738979401cb3f70128c38ef826'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:03.729621 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1178/2348] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:17:03 2022] DELETE /identity/v3/users/2978b4738979401cb3f70128c38ef826 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:17:04.004579 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96c581-1f57-4e52-b2c4-fdd83a1dd9b9 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:04.006129 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-ab96c581-1f57-4e52-b2c4-fdd83a1dd9b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:04.025098 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96c581-1f57-4e52-b2c4-fdd83a1dd9b9 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:04.083998 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6e45d-f876-42c6-a8ef-b8701cfc1241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:04.084623 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6e45d-f876-42c6-a8ef-b8701cfc1241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:04.085016 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6e45d-f876-42c6-a8ef-b8701cfc1241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:04.085299 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6e45d-f876-42c6-a8ef-b8701cfc1241 admin admin] PATH_INFO: `/v3/projects/37031c4080a14e45aa7531e20394c6fb` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:04.086402 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6e45d-f876-42c6-a8ef-b8701cfc1241 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37031c4080a14e45aa7531e20394c6fb)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:04.088362 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6e45d-f876-42c6-a8ef-b8701cfc1241 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:04.137204 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:17:04.140518 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-4ae6e45d-f876-42c6-a8ef-b8701cfc1241 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37031c4080a14e45aa7531e20394c6fb', 'request_id': 'req-4ae6e45d-f876-42c6-a8ef-b8701cfc1241'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:04.148177 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-4ae6e45d-f876-42c6-a8ef-b8701cfc1241 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37031c4080a14e45aa7531e20394c6fb', 'request_id': 'req-4ae6e45d-f876-42c6-a8ef-b8701cfc1241'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:04.156505 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-4ae6e45d-f876-42c6-a8ef-b8701cfc1241 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37031c4080a14e45aa7531e20394c6fb', 'request_id': 'req-4ae6e45d-f876-42c6-a8ef-b8701cfc1241'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:04.176921 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-4ae6e45d-f876-42c6-a8ef-b8701cfc1241 admin admin] The token cache is being invalidate because project 37031c4080a14e45aa7531e20394c6fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:17:04.177495 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-4ae6e45d-f876-42c6-a8ef-b8701cfc1241 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:04.179222 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-4ae6e45d-f876-42c6-a8ef-b8701cfc1241 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:04.181006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1171/2349] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:17:04 2022] DELETE /identity/v3/projects/37031c4080a14e45aa7531e20394c6fb => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:17:04.185007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8ef15-78f3-4f6f-9dc1-ba7061a73b9f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:04.186571 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-78f8ef15-78f3-4f6f-9dc1-ba7061a73b9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:04.200652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f8ef15-78f3-4f6f-9dc1-ba7061a73b9f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:04.269920 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300090d9-4a75-4cd9-9183-9d7900af4a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:04.270712 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300090d9-4a75-4cd9-9183-9d7900af4a5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:04.271073 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300090d9-4a75-4cd9-9183-9d7900af4a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:04.271438 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300090d9-4a75-4cd9-9183-9d7900af4a5b admin admin] PATH_INFO: `/v3/users/b0758314c4ab4cfd97f31d72e947fe52` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:04.272648 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300090d9-4a75-4cd9-9183-9d7900af4a5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0758314c4ab4cfd97f31d72e947fe52)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:04.274707 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300090d9-4a75-4cd9-9183-9d7900af4a5b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:04.326198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-300090d9-4a75-4cd9-9183-9d7900af4a5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0758314c4ab4cfd97f31d72e947fe52'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:04.333558 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-300090d9-4a75-4cd9-9183-9d7900af4a5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0758314c4ab4cfd97f31d72e947fe52'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:04.334325 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-300090d9-4a75-4cd9-9183-9d7900af4a5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0758314c4ab4cfd97f31d72e947fe52'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:04.342966 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-300090d9-4a75-4cd9-9183-9d7900af4a5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0758314c4ab4cfd97f31d72e947fe52'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:04.355198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-300090d9-4a75-4cd9-9183-9d7900af4a5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0758314c4ab4cfd97f31d72e947fe52'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:04.357166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:17:04.365375 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1179/2350] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:17:04 2022] DELETE /identity/v3/users/b0758314c4ab4cfd97f31d72e947fe52 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:17:04.833683 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6e45d-f876-42c6-a8ef-b8701cfc1241 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:04.835326 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-4ae6e45d-f876-42c6-a8ef-b8701cfc1241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:04.850603 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6e45d-f876-42c6-a8ef-b8701cfc1241 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:04.918263 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8312c117-3d5b-4e56-be70-2810af83bd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:04.918986 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8312c117-3d5b-4e56-be70-2810af83bd83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:04.919366 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8312c117-3d5b-4e56-be70-2810af83bd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:04.919725 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8312c117-3d5b-4e56-be70-2810af83bd83 admin admin] PATH_INFO: `/v3/projects/77635c4d233345edb8736b4fbe3d8cb8` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:04.924555 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8312c117-3d5b-4e56-be70-2810af83bd83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77635c4d233345edb8736b4fbe3d8cb8)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:04.926930 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8312c117-3d5b-4e56-be70-2810af83bd83 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:04.979669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-8312c117-3d5b-4e56-be70-2810af83bd83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77635c4d233345edb8736b4fbe3d8cb8', 'request_id': 'req-8312c117-3d5b-4e56-be70-2810af83bd83'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:04.985229 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-8312c117-3d5b-4e56-be70-2810af83bd83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77635c4d233345edb8736b4fbe3d8cb8', 'request_id': 'req-8312c117-3d5b-4e56-be70-2810af83bd83'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:04.991549 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-8312c117-3d5b-4e56-be70-2810af83bd83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77635c4d233345edb8736b4fbe3d8cb8', 'request_id': 'req-8312c117-3d5b-4e56-be70-2810af83bd83'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:05.005691 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-8312c117-3d5b-4e56-be70-2810af83bd83 admin admin] The token cache is being invalidate because project 77635c4d233345edb8736b4fbe3d8cb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:17:05.006319 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-8312c117-3d5b-4e56-be70-2810af83bd83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:05.006605 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-8312c117-3d5b-4e56-be70-2810af83bd83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:05.008157 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1172/2351] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:17:04 2022] DELETE /identity/v3/projects/77635c4d233345edb8736b4fbe3d8cb8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:17:05.014984 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54ddc2-e873-45e8-bbe6-796292dac84a None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:05.015532 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54ddc2-e873-45e8-bbe6-796292dac84a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:05.015631 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54ddc2-e873-45e8-bbe6-796292dac84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:05.039172 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8c54ddc2-e873-45e8-bbe6-796292dac84a None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:17:05.040498 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8c54ddc2-e873-45e8-bbe6-796292dac84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:05.102570 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1180/2352] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:17:05.111369 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec887e7e-abdb-44fe-b9c0-c69918a2eae2 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:05.111767 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec887e7e-abdb-44fe-b9c0-c69918a2eae2 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:05.112107 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec887e7e-abdb-44fe-b9c0-c69918a2eae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:05.132580 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-ec887e7e-abdb-44fe-b9c0-c69918a2eae2 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:17:05.133860 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-ec887e7e-abdb-44fe-b9c0-c69918a2eae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:05.142495 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1173/2353] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:17:05.146669 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54ddc2-e873-45e8-bbe6-796292dac84a None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:05.156711 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54ddc2-e873-45e8-bbe6-796292dac84a None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:05.161446 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b29c6-4425-47da-8531-c996ade41fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:05.162044 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b29c6-4425-47da-8531-c996ade41fd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:05.162321 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b29c6-4425-47da-8531-c996ade41fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:05.162591 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b29c6-4425-47da-8531-c996ade41fd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:05.163333 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b29c6-4425-47da-8531-c996ade41fd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:17:05.163635 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b29c6-4425-47da-8531-c996ade41fd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:05.165215 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b29c6-4425-47da-8531-c996ade41fd0 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:05.177103 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1181/2354] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:17:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:17:05.181778 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec887e7e-abdb-44fe-b9c0-c69918a2eae2 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:05.191722 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec887e7e-abdb-44fe-b9c0-c69918a2eae2 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:05.196435 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36caa245-874b-485f-9e06-ba36ec1166bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:05.197035 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caa245-874b-485f-9e06-ba36ec1166bc admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:05.197313 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caa245-874b-485f-9e06-ba36ec1166bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:05.197582 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caa245-874b-485f-9e06-ba36ec1166bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:05.198240 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36caa245-874b-485f-9e06-ba36ec1166bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:05.200798 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36caa245-874b-485f-9e06-ba36ec1166bc admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:05.228118 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1174/2355] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:17:05 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:17:05.233213 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b29c6-4425-47da-8531-c996ade41fd0 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:05.243083 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b29c6-4425-47da-8531-c996ade41fd0 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:05.300693 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374f049-599e-4ab1-9413-6810737417fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:05.301300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374f049-599e-4ab1-9413-6810737417fd admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:05.301579 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374f049-599e-4ab1-9413-6810737417fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:05.301972 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374f049-599e-4ab1-9413-6810737417fd admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:05.302607 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b374f049-599e-4ab1-9413-6810737417fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:05.304588 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b374f049-599e-4ab1-9413-6810737417fd admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:05.335253 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1182/2356] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:17:05 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:17:05.339449 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36caa245-874b-485f-9e06-ba36ec1166bc admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:05.350792 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36caa245-874b-485f-9e06-ba36ec1166bc admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:05.356425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f446c-12e0-4827-b81b-b91f7613ad70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:05.357083 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f446c-12e0-4827-b81b-b91f7613ad70 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:05.357457 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f446c-12e0-4827-b81b-b91f7613ad70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:05.358348 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f446c-12e0-4827-b81b-b91f7613ad70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:05.358581 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f446c-12e0-4827-b81b-b91f7613ad70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:05.360331 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f446c-12e0-4827-b81b-b91f7613ad70 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:05.372162 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1175/2357] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:17:05.376118 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374f049-599e-4ab1-9413-6810737417fd admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:05.386756 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374f049-599e-4ab1-9413-6810737417fd admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:05.393932 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a3f56-d1a7-4e44-9b63-eac336642ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:05.394983 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a3f56-d1a7-4e44-9b63-eac336642ece admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:05.395357 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a3f56-d1a7-4e44-9b63-eac336642ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:05.395698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a3f56-d1a7-4e44-9b63-eac336642ece admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:05.396418 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024a3f56-d1a7-4e44-9b63-eac336642ece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:05.398112 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024a3f56-d1a7-4e44-9b63-eac336642ece admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:05.413590 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1183/2358] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:17:05.414365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f446c-12e0-4827-b81b-b91f7613ad70 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:05.426099 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f446c-12e0-4827-b81b-b91f7613ad70 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:05.430731 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df167c-a9ae-420f-aac9-b81537323a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:05.431405 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df167c-a9ae-420f-aac9-b81537323a02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:05.431685 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df167c-a9ae-420f-aac9-b81537323a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:05.431956 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df167c-a9ae-420f-aac9-b81537323a02 admin admin] PATH_INFO: `/v3/projects/adb6f21fac174d77bd9313dc426b8e45/users/bb6cef40914c4f1cb5544bb5e5da4428/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:05.448289 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df167c-a9ae-420f-aac9-b81537323a02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adb6f21fac174d77bd9313dc426b8e45, user_id=bb6cef40914c4f1cb5544bb5e5da4428, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:05.451502 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df167c-a9ae-420f-aac9-b81537323a02 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:05.460297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:17:05.465883 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1176/2359] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:17:05 2022] PUT /identity/v3/projects/adb6f21fac174d77bd9313dc426b8e45/users/bb6cef40914c4f1cb5544bb5e5da4428/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:17:05.478542 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5bca7-b477-4815-bc7b-949d2b393dcc None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:05.478847 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5bca7-b477-4815-bc7b-949d2b393dcc None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:05.479142 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5bca7-b477-4815-bc7b-949d2b393dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:05.506640 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d2b5bca7-b477-4815-bc7b-949d2b393dcc None None] MFA Rules not processed for user `bb6cef40914c4f1cb5544bb5e5da4428`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:17:05.507888 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d2b5bca7-b477-4815-bc7b-949d2b393dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:05.560731 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:17:05.571808 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1184/2360] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:17:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:17:05.583863 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df167c-a9ae-420f-aac9-b81537323a02 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:05.585567 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-78df167c-a9ae-420f-aac9-b81537323a02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:05.641299 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df167c-a9ae-420f-aac9-b81537323a02 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:05.644819 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ea091-4926-4a13-865d-ef8128c9f5f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:05.645411 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ea091-4926-4a13-865d-ef8128c9f5f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:05.645689 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ea091-4926-4a13-865d-ef8128c9f5f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:05.645959 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ea091-4926-4a13-865d-ef8128c9f5f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:05.654044 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ea091-4926-4a13-865d-ef8128c9f5f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:05.655496 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ea091-4926-4a13-865d-ef8128c9f5f2 service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:05.663970 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1177/2361] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:17:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:17:05.821464 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5bca7-b477-4815-bc7b-949d2b393dcc None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:05.822603 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d2b5bca7-b477-4815-bc7b-949d2b393dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:05.906371 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5bca7-b477-4815-bc7b-949d2b393dcc None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:05.913103 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad7150c-3e5c-45dc-b51a-ce6b331ea56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:05.913855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7150c-3e5c-45dc-b51a-ce6b331ea56b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:05.914135 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7150c-3e5c-45dc-b51a-ce6b331ea56b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:05.914408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7150c-3e5c-45dc-b51a-ce6b331ea56b admin admin] PATH_INFO: `/v3/users/61c90e0f6600441883fc81ea1650e7b2` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:05.915557 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad7150c-3e5c-45dc-b51a-ce6b331ea56b admin admin] RBAC: Authorizing `identity:delete_user(user_id=61c90e0f6600441883fc81ea1650e7b2)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:05.918121 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad7150c-3e5c-45dc-b51a-ce6b331ea56b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:05.990089 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0ad7150c-3e5c-45dc-b51a-ce6b331ea56b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61c90e0f6600441883fc81ea1650e7b2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:06.000427 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0ad7150c-3e5c-45dc-b51a-ce6b331ea56b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61c90e0f6600441883fc81ea1650e7b2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:06.001212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0ad7150c-3e5c-45dc-b51a-ce6b331ea56b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61c90e0f6600441883fc81ea1650e7b2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:06.013193 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0ad7150c-3e5c-45dc-b51a-ce6b331ea56b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61c90e0f6600441883fc81ea1650e7b2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:06.031778 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0ad7150c-3e5c-45dc-b51a-ce6b331ea56b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61c90e0f6600441883fc81ea1650e7b2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:06.041053 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1185/2362] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:17:05 2022] DELETE /identity/v3/users/61c90e0f6600441883fc81ea1650e7b2 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:17:06.044037 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db37a6-18b3-4df7-b99f-6181565e1f14 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:06.044524 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db37a6-18b3-4df7-b99f-6181565e1f14 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:06.044620 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db37a6-18b3-4df7-b99f-6181565e1f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:06.064856 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-45db37a6-18b3-4df7-b99f-6181565e1f14 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:17:06.066148 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-45db37a6-18b3-4df7-b99f-6181565e1f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:06.123777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1178/2363] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:17:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:17:06.133470 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad7150c-3e5c-45dc-b51a-ce6b331ea56b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:06.134839 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0ad7150c-3e5c-45dc-b51a-ce6b331ea56b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:06.191248 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad7150c-3e5c-45dc-b51a-ce6b331ea56b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:06.194869 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02d707-e7a0-4aa7-a7ef-eeb0a199de23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:06.195494 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02d707-e7a0-4aa7-a7ef-eeb0a199de23 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:06.195773 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02d707-e7a0-4aa7-a7ef-eeb0a199de23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:06.196039 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02d707-e7a0-4aa7-a7ef-eeb0a199de23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:06.205105 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e02d707-e7a0-4aa7-a7ef-eeb0a199de23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:06.206518 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e02d707-e7a0-4aa7-a7ef-eeb0a199de23 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:06.215956 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1186/2364] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:17:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:17:06.489117 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45db37a6-18b3-4df7-b99f-6181565e1f14 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:06.490583 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-45db37a6-18b3-4df7-b99f-6181565e1f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:06.508276 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45db37a6-18b3-4df7-b99f-6181565e1f14 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:06.514915 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea55f371-56ff-46ed-ade2-507a101a4d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:06.526104 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55f371-56ff-46ed-ade2-507a101a4d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:06.526496 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55f371-56ff-46ed-ade2-507a101a4d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:06.526843 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55f371-56ff-46ed-ade2-507a101a4d05 admin admin] PATH_INFO: `/v3/projects/7913b472c713462daa5945cd12c82cf6` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:06.528367 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea55f371-56ff-46ed-ade2-507a101a4d05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7913b472c713462daa5945cd12c82cf6)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:06.530360 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea55f371-56ff-46ed-ade2-507a101a4d05 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:06.581084 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-ea55f371-56ff-46ed-ade2-507a101a4d05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7913b472c713462daa5945cd12c82cf6', 'request_id': 'req-ea55f371-56ff-46ed-ade2-507a101a4d05'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:06.586838 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-ea55f371-56ff-46ed-ade2-507a101a4d05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7913b472c713462daa5945cd12c82cf6', 'request_id': 'req-ea55f371-56ff-46ed-ade2-507a101a4d05'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:06.593039 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-ea55f371-56ff-46ed-ade2-507a101a4d05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7913b472c713462daa5945cd12c82cf6', 'request_id': 'req-ea55f371-56ff-46ed-ade2-507a101a4d05'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:06.606888 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-ea55f371-56ff-46ed-ade2-507a101a4d05 admin admin] The token cache is being invalidate because project 7913b472c713462daa5945cd12c82cf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:17:06.607212 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-ea55f371-56ff-46ed-ade2-507a101a4d05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:06.607855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-ea55f371-56ff-46ed-ade2-507a101a4d05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:06.609327 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1179/2365] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:17:06 2022] DELETE /identity/v3/projects/7913b472c713462daa5945cd12c82cf6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:17:06.614370 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc032d-59cc-4c39-aa5e-005bb249eb6d None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:06.614654 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc032d-59cc-4c39-aa5e-005bb249eb6d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:06.614929 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc032d-59cc-4c39-aa5e-005bb249eb6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:06.634802 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-cddc032d-59cc-4c39-aa5e-005bb249eb6d None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:17:06.636019 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-cddc032d-59cc-4c39-aa5e-005bb249eb6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:06.645724 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:17:06.696849 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1187/2366] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:17:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:17:06.760925 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e864124-7b49-4285-9a18-40b92546e2f8 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:06.761206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e864124-7b49-4285-9a18-40b92546e2f8 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:06.761485 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e864124-7b49-4285-9a18-40b92546e2f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:06.780894 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-0e864124-7b49-4285-9a18-40b92546e2f8 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:17:06.782086 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-0e864124-7b49-4285-9a18-40b92546e2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:06.789901 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1180/2367] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:17:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:17:06.794684 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc032d-59cc-4c39-aa5e-005bb249eb6d None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:06.804669 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc032d-59cc-4c39-aa5e-005bb249eb6d None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:06.809894 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1124359-cded-4f20-bb36-dee02ea4acff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:06.810490 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1124359-cded-4f20-bb36-dee02ea4acff admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:06.810780 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1124359-cded-4f20-bb36-dee02ea4acff admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:06.811068 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1124359-cded-4f20-bb36-dee02ea4acff admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:06.811738 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1124359-cded-4f20-bb36-dee02ea4acff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:17:06.812041 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1124359-cded-4f20-bb36-dee02ea4acff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:06.813627 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1124359-cded-4f20-bb36-dee02ea4acff admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:06.824848 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1188/2368] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:17:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:17:06.829049 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e864124-7b49-4285-9a18-40b92546e2f8 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:06.838109 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e864124-7b49-4285-9a18-40b92546e2f8 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:06.842948 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea99c1e-b7f6-48ec-8897-914e3d95a4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:06.843596 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea99c1e-b7f6-48ec-8897-914e3d95a4f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:06.843868 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea99c1e-b7f6-48ec-8897-914e3d95a4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:06.844134 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea99c1e-b7f6-48ec-8897-914e3d95a4f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:06.844769 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea99c1e-b7f6-48ec-8897-914e3d95a4f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:06.846648 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea99c1e-b7f6-48ec-8897-914e3d95a4f8 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:06.866615 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:17:06.878141 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1181/2369] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:17:06 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:17:06.879261 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1124359-cded-4f20-bb36-dee02ea4acff admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:06.889318 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1124359-cded-4f20-bb36-dee02ea4acff admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:06.946446 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb150b5-1177-4d6b-bb82-7a51991614df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:06.947039 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb150b5-1177-4d6b-bb82-7a51991614df admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:06.947340 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb150b5-1177-4d6b-bb82-7a51991614df admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:06.947608 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb150b5-1177-4d6b-bb82-7a51991614df admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:06.948236 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb150b5-1177-4d6b-bb82-7a51991614df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:06.950182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb150b5-1177-4d6b-bb82-7a51991614df admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:06.970350 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1189/2370] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:17:06 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:17:06.974558 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea99c1e-b7f6-48ec-8897-914e3d95a4f8 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:06.985501 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea99c1e-b7f6-48ec-8897-914e3d95a4f8 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:06.990122 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d88862-6533-4e99-acdd-42ae16b445c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:06.990894 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d88862-6533-4e99-acdd-42ae16b445c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:06.991218 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d88862-6533-4e99-acdd-42ae16b445c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:06.991493 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d88862-6533-4e99-acdd-42ae16b445c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:06.992134 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d88862-6533-4e99-acdd-42ae16b445c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:06.993724 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d88862-6533-4e99-acdd-42ae16b445c0 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:07.004088 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1182/2371] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:17:07.007835 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb150b5-1177-4d6b-bb82-7a51991614df admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:07.017862 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb150b5-1177-4d6b-bb82-7a51991614df admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:07.022529 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942f85ed-f793-465d-b7be-a062491b50ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:07.023158 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f85ed-f793-465d-b7be-a062491b50ed admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:07.023440 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f85ed-f793-465d-b7be-a062491b50ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:07.023711 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f85ed-f793-465d-b7be-a062491b50ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:07.024427 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942f85ed-f793-465d-b7be-a062491b50ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:07.025900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942f85ed-f793-465d-b7be-a062491b50ed admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:07.036245 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1190/2372] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:17:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:17:07.040353 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d88862-6533-4e99-acdd-42ae16b445c0 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:07.051570 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d88862-6533-4e99-acdd-42ae16b445c0 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:07.056401 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99627562-7f93-4b58-9d49-484d6342e468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:07.057136 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99627562-7f93-4b58-9d49-484d6342e468 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:07.057485 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99627562-7f93-4b58-9d49-484d6342e468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:07.057824 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99627562-7f93-4b58-9d49-484d6342e468 admin admin] PATH_INFO: `/v3/projects/d54fa579844f4ba1aabc1c3009d31d11/users/a285c38d4fd743629fc5e64eb52af8a1/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:07.073177 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99627562-7f93-4b58-9d49-484d6342e468 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d54fa579844f4ba1aabc1c3009d31d11, user_id=a285c38d4fd743629fc5e64eb52af8a1, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:07.076080 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99627562-7f93-4b58-9d49-484d6342e468 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:07.089934 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1183/2373] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:17:07 2022] PUT /identity/v3/projects/d54fa579844f4ba1aabc1c3009d31d11/users/a285c38d4fd743629fc5e64eb52af8a1/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:17:07.100000 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811c35b-cb14-473c-8570-855e10d95f04 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:07.100388 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811c35b-cb14-473c-8570-855e10d95f04 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:07.100642 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811c35b-cb14-473c-8570-855e10d95f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:07.129902 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-3811c35b-cb14-473c-8570-855e10d95f04 None None] MFA Rules not processed for user `a285c38d4fd743629fc5e64eb52af8a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:17:07.131275 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3811c35b-cb14-473c-8570-855e10d95f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:07.197885 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1191/2374] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:17:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:17:07.211296 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99627562-7f93-4b58-9d49-484d6342e468 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:07.211519 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-99627562-7f93-4b58-9d49-484d6342e468 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:07.273019 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99627562-7f93-4b58-9d49-484d6342e468 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:07.276540 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936df69e-af08-48f8-8e9c-76e2ba4f6f67 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45ce234c5b4643b2ac6341e0928f3830', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:07.277677 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936df69e-af08-48f8-8e9c-76e2ba4f6f67 service glance] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:07.277955 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936df69e-af08-48f8-8e9c-76e2ba4f6f67 service glance] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:07.278225 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936df69e-af08-48f8-8e9c-76e2ba4f6f67 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:07.286674 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936df69e-af08-48f8-8e9c-76e2ba4f6f67 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:07.288144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936df69e-af08-48f8-8e9c-76e2ba4f6f67 service glance] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:07.297353 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1184/2375] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:17:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 13 13:17:17.194876 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:17:17.292813 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=42,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:17:25.459401 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3811c35b-cb14-473c-8570-855e10d95f04 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:25.460837 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3811c35b-cb14-473c-8570-855e10d95f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:25.526606 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3811c35b-cb14-473c-8570-855e10d95f04 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:25.531168 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed294bc1-1383-476d-80e3-f54882453d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:25.531793 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed294bc1-1383-476d-80e3-f54882453d9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:25.532073 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed294bc1-1383-476d-80e3-f54882453d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:25.532351 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed294bc1-1383-476d-80e3-f54882453d9c admin admin] PATH_INFO: `/v3/users/bb6cef40914c4f1cb5544bb5e5da4428` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:25.533484 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed294bc1-1383-476d-80e3-f54882453d9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb6cef40914c4f1cb5544bb5e5da4428)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:25.536033 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed294bc1-1383-476d-80e3-f54882453d9c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:25.582591 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ed294bc1-1383-476d-80e3-f54882453d9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb6cef40914c4f1cb5544bb5e5da4428'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:25.589344 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ed294bc1-1383-476d-80e3-f54882453d9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb6cef40914c4f1cb5544bb5e5da4428'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:25.589943 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ed294bc1-1383-476d-80e3-f54882453d9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb6cef40914c4f1cb5544bb5e5da4428'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:25.597675 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ed294bc1-1383-476d-80e3-f54882453d9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb6cef40914c4f1cb5544bb5e5da4428'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:25.609839 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ed294bc1-1383-476d-80e3-f54882453d9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb6cef40914c4f1cb5544bb5e5da4428'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:25.615826 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1192/2376] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:17:25 2022] DELETE /identity/v3/users/bb6cef40914c4f1cb5544bb5e5da4428 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:17:25.620365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e64ea-1470-491d-b828-05ee39464cc6 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:25.620832 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e64ea-1470-491d-b828-05ee39464cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:25.620947 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e64ea-1470-491d-b828-05ee39464cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:25.640150 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-042e64ea-1470-491d-b828-05ee39464cc6 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:17:25.641347 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-042e64ea-1470-491d-b828-05ee39464cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:25.705274 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1185/2377] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:17:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:17:25.712465 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed294bc1-1383-476d-80e3-f54882453d9c admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:25.713855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ed294bc1-1383-476d-80e3-f54882453d9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:25.773075 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed294bc1-1383-476d-80e3-f54882453d9c admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:25.776857 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bcfbac-d796-43ce-b8e8-3f9ab5c86c76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:25.777623 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcfbac-d796-43ce-b8e8-3f9ab5c86c76 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:25.777904 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcfbac-d796-43ce-b8e8-3f9ab5c86c76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:25.778198 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcfbac-d796-43ce-b8e8-3f9ab5c86c76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:25.789248 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bcfbac-d796-43ce-b8e8-3f9ab5c86c76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:25.790810 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bcfbac-d796-43ce-b8e8-3f9ab5c86c76 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:25.801180 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1193/2378] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:17:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 13 13:17:26.071309 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e64ea-1470-491d-b828-05ee39464cc6 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:26.072763 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-042e64ea-1470-491d-b828-05ee39464cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:26.085854 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e64ea-1470-491d-b828-05ee39464cc6 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:26.090441 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a2fa8-3370-4676-8413-e3925768e98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:26.091104 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a2fa8-3370-4676-8413-e3925768e98a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:26.091458 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a2fa8-3370-4676-8413-e3925768e98a admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:26.091732 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a2fa8-3370-4676-8413-e3925768e98a admin admin] PATH_INFO: `/v3/projects/adb6f21fac174d77bd9313dc426b8e45` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:26.092842 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368a2fa8-3370-4676-8413-e3925768e98a admin admin] RBAC: Authorizing `identity:delete_project(project_id=adb6f21fac174d77bd9313dc426b8e45)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:26.094754 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368a2fa8-3370-4676-8413-e3925768e98a admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:26.141300 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-368a2fa8-3370-4676-8413-e3925768e98a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adb6f21fac174d77bd9313dc426b8e45', 'request_id': 'req-368a2fa8-3370-4676-8413-e3925768e98a'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:26.146711 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-368a2fa8-3370-4676-8413-e3925768e98a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adb6f21fac174d77bd9313dc426b8e45', 'request_id': 'req-368a2fa8-3370-4676-8413-e3925768e98a'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:26.153739 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-368a2fa8-3370-4676-8413-e3925768e98a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adb6f21fac174d77bd9313dc426b8e45', 'request_id': 'req-368a2fa8-3370-4676-8413-e3925768e98a'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:26.165966 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-368a2fa8-3370-4676-8413-e3925768e98a admin admin] The token cache is being invalidate because project adb6f21fac174d77bd9313dc426b8e45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:17:26.166283 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-368a2fa8-3370-4676-8413-e3925768e98a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:26.166900 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-368a2fa8-3370-4676-8413-e3925768e98a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:26.168406 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1186/2379] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:17:26 2022] DELETE /identity/v3/projects/adb6f21fac174d77bd9313dc426b8e45 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:17:26.173657 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25aaa2b-e054-4928-ae72-c0e28ff2938c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:26.173944 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25aaa2b-e054-4928-ae72-c0e28ff2938c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:26.174218 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25aaa2b-e054-4928-ae72-c0e28ff2938c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:26.194389 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c25aaa2b-e054-4928-ae72-c0e28ff2938c None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:17:26.195622 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c25aaa2b-e054-4928-ae72-c0e28ff2938c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:26.211566 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:17:26.254495 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1194/2380] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:17:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:17:26.262360 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8536fc-af83-4627-9646-fc8559a35145 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:26.262662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8536fc-af83-4627-9646-fc8559a35145 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:26.262933 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8536fc-af83-4627-9646-fc8559a35145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:26.282830 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-0a8536fc-af83-4627-9646-fc8559a35145 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:17:26.284043 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-0a8536fc-af83-4627-9646-fc8559a35145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:26.292073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1187/2381] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:17:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:17:26.296123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25aaa2b-e054-4928-ae72-c0e28ff2938c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:26.306297 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25aaa2b-e054-4928-ae72-c0e28ff2938c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:26.311076 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb13cc-b4ae-4fe5-a14f-1ae4ecf922ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:26.311687 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb13cc-b4ae-4fe5-a14f-1ae4ecf922ea admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:26.311969 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb13cc-b4ae-4fe5-a14f-1ae4ecf922ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:26.312357 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb13cc-b4ae-4fe5-a14f-1ae4ecf922ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:26.313008 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb13cc-b4ae-4fe5-a14f-1ae4ecf922ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:17:26.313334 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb13cc-b4ae-4fe5-a14f-1ae4ecf922ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:26.314925 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb13cc-b4ae-4fe5-a14f-1ae4ecf922ea admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:26.326922 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1195/2382] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:17:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:17:26.331265 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8536fc-af83-4627-9646-fc8559a35145 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:26.340234 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8536fc-af83-4627-9646-fc8559a35145 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:26.344866 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089e14a-ffc6-43a1-b565-8656ab554ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:26.345476 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089e14a-ffc6-43a1-b565-8656ab554ec8 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:26.345754 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089e14a-ffc6-43a1-b565-8656ab554ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:26.346023 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089e14a-ffc6-43a1-b565-8656ab554ec8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:26.346720 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089e14a-ffc6-43a1-b565-8656ab554ec8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:26.348613 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5089e14a-ffc6-43a1-b565-8656ab554ec8 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:26.374793 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1188/2383] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:17:26 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:17:26.379565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb13cc-b4ae-4fe5-a14f-1ae4ecf922ea admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:26.389462 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb13cc-b4ae-4fe5-a14f-1ae4ecf922ea admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:26.444004 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d49859-814d-4c69-abd0-f8d5d3f23774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:26.444531 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d49859-814d-4c69-abd0-f8d5d3f23774 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:26.444812 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d49859-814d-4c69-abd0-f8d5d3f23774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:26.445084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d49859-814d-4c69-abd0-f8d5d3f23774 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:26.445714 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d49859-814d-4c69-abd0-f8d5d3f23774 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:26.447656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d49859-814d-4c69-abd0-f8d5d3f23774 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:26.467829 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1196/2384] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:17:26 2022] POST /identity/v3/users => generated 288 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:17:26.472501 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089e14a-ffc6-43a1-b565-8656ab554ec8 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:26.482571 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5089e14a-ffc6-43a1-b565-8656ab554ec8 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:26.487129 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99bf8a-c56a-4477-8dc7-1ac4cfe60694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:26.488354 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99bf8a-c56a-4477-8dc7-1ac4cfe60694 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:26.488635 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99bf8a-c56a-4477-8dc7-1ac4cfe60694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:26.488904 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99bf8a-c56a-4477-8dc7-1ac4cfe60694 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:26.489535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c99bf8a-c56a-4477-8dc7-1ac4cfe60694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:26.491151 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c99bf8a-c56a-4477-8dc7-1ac4cfe60694 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:26.501237 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1189/2385] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:17:26.504949 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d49859-814d-4c69-abd0-f8d5d3f23774 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:26.514631 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d49859-814d-4c69-abd0-f8d5d3f23774 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:26.519280 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d2faee-827d-4c3b-a56b-c4b4215d7734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:26.519867 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2faee-827d-4c3b-a56b-c4b4215d7734 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:26.520150 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2faee-827d-4c3b-a56b-c4b4215d7734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:26.520431 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2faee-827d-4c3b-a56b-c4b4215d7734 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:26.521067 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d2faee-827d-4c3b-a56b-c4b4215d7734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:26.522669 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d2faee-827d-4c3b-a56b-c4b4215d7734 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:26.535814 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1197/2386] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:17:26.538958 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99bf8a-c56a-4477-8dc7-1ac4cfe60694 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:26.549123 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99bf8a-c56a-4477-8dc7-1ac4cfe60694 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:26.554277 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b3341-deff-4301-b129-359543c81713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:26.554921 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b3341-deff-4301-b129-359543c81713 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:26.555236 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b3341-deff-4301-b129-359543c81713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:26.555509 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b3341-deff-4301-b129-359543c81713 admin admin] PATH_INFO: `/v3/projects/28f27dc791d6469cb0432709d7242c81/users/e86c98aa51f4412fb688f2c8ffbf0d54/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:26.569565 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214b3341-deff-4301-b129-359543c81713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28f27dc791d6469cb0432709d7242c81, user_id=e86c98aa51f4412fb688f2c8ffbf0d54, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:26.572260 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214b3341-deff-4301-b129-359543c81713 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:26.584967 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1190/2387] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:17:26 2022] PUT /identity/v3/projects/28f27dc791d6469cb0432709d7242c81/users/e86c98aa51f4412fb688f2c8ffbf0d54/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:17:26.595191 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f29f968-183c-4c3a-b3bd-2d10eed3f64f None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:26.595484 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f29f968-183c-4c3a-b3bd-2d10eed3f64f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:26.595780 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f29f968-183c-4c3a-b3bd-2d10eed3f64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:26.623565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8f29f968-183c-4c3a-b3bd-2d10eed3f64f None None] MFA Rules not processed for user `e86c98aa51f4412fb688f2c8ffbf0d54`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:17:26.624766 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8f29f968-183c-4c3a-b3bd-2d10eed3f64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:26.688579 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1198/2388] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:17:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:17:26.699727 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b3341-deff-4301-b129-359543c81713 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:26.701330 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-214b3341-deff-4301-b129-359543c81713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:26.723137 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:17:26.759308 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b3341-deff-4301-b129-359543c81713 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:26.762917 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4b51d-23bb-4f77-82f2-b6ea1ac01b2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:26.763644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4b51d-23bb-4f77-82f2-b6ea1ac01b2e service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:26.763926 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4b51d-23bb-4f77-82f2-b6ea1ac01b2e service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:26.764197 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4b51d-23bb-4f77-82f2-b6ea1ac01b2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:26.772452 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4b51d-23bb-4f77-82f2-b6ea1ac01b2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:26.773869 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4b51d-23bb-4f77-82f2-b6ea1ac01b2e service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:26.783937 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1191/2389] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:17:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:17:36.177295 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f29f968-183c-4c3a-b3bd-2d10eed3f64f None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:36.178671 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8f29f968-183c-4c3a-b3bd-2d10eed3f64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:36.197704 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:17:36.243802 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f29f968-183c-4c3a-b3bd-2d10eed3f64f None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:36.248233 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7d49f-1326-4ad8-8f31-aabe977f27b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:36.248846 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7d49f-1326-4ad8-8f31-aabe977f27b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:36.249126 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7d49f-1326-4ad8-8f31-aabe977f27b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:36.249397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7d49f-1326-4ad8-8f31-aabe977f27b4 admin admin] PATH_INFO: `/v3/users/a285c38d4fd743629fc5e64eb52af8a1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:36.250491 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d7d49f-1326-4ad8-8f31-aabe977f27b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a285c38d4fd743629fc5e64eb52af8a1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:36.252429 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d7d49f-1326-4ad8-8f31-aabe977f27b4 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:36.296313 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-44d7d49f-1326-4ad8-8f31-aabe977f27b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a285c38d4fd743629fc5e64eb52af8a1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:36.303912 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-44d7d49f-1326-4ad8-8f31-aabe977f27b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a285c38d4fd743629fc5e64eb52af8a1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:36.304545 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-44d7d49f-1326-4ad8-8f31-aabe977f27b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a285c38d4fd743629fc5e64eb52af8a1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:36.311452 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-44d7d49f-1326-4ad8-8f31-aabe977f27b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a285c38d4fd743629fc5e64eb52af8a1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:36.322880 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-44d7d49f-1326-4ad8-8f31-aabe977f27b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a285c38d4fd743629fc5e64eb52af8a1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:36.328565 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1199/2390] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:17:36 2022] DELETE /identity/v3/users/a285c38d4fd743629fc5e64eb52af8a1 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:17:36.333213 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f96ff-b5c6-4ae3-b5bc-4282a7692a84 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:36.333707 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f96ff-b5c6-4ae3-b5bc-4282a7692a84 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:36.334534 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f96ff-b5c6-4ae3-b5bc-4282a7692a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:36.353538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-707f96ff-b5c6-4ae3-b5bc-4282a7692a84 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:17:36.354719 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-707f96ff-b5c6-4ae3-b5bc-4282a7692a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:36.411878 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1192/2391] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:17:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:17:36.421554 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7d49f-1326-4ad8-8f31-aabe977f27b4 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:36.423244 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-44d7d49f-1326-4ad8-8f31-aabe977f27b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:36.478561 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7d49f-1326-4ad8-8f31-aabe977f27b4 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:36.482196 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacbe2b-5413-470a-aab1-78ae5f320042 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:36.482791 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacbe2b-5413-470a-aab1-78ae5f320042 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:36.483092 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacbe2b-5413-470a-aab1-78ae5f320042 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:36.483372 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacbe2b-5413-470a-aab1-78ae5f320042 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:36.492633 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bacbe2b-5413-470a-aab1-78ae5f320042 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:36.494049 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bacbe2b-5413-470a-aab1-78ae5f320042 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:36.503309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1200/2392] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:17:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:17:36.803800 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f96ff-b5c6-4ae3-b5bc-4282a7692a84 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:36.805407 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-707f96ff-b5c6-4ae3-b5bc-4282a7692a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:36.821092 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f96ff-b5c6-4ae3-b5bc-4282a7692a84 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:36.826671 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae67b6f-80e6-4e68-b182-738101861894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:36.827350 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae67b6f-80e6-4e68-b182-738101861894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:36.827714 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae67b6f-80e6-4e68-b182-738101861894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:36.827900 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae67b6f-80e6-4e68-b182-738101861894 admin admin] PATH_INFO: `/v3/projects/d54fa579844f4ba1aabc1c3009d31d11` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:36.829123 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae67b6f-80e6-4e68-b182-738101861894 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d54fa579844f4ba1aabc1c3009d31d11)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:36.831030 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae67b6f-80e6-4e68-b182-738101861894 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:36.876966 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-1ae67b6f-80e6-4e68-b182-738101861894 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd54fa579844f4ba1aabc1c3009d31d11', 'request_id': 'req-1ae67b6f-80e6-4e68-b182-738101861894'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:36.882321 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-1ae67b6f-80e6-4e68-b182-738101861894 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd54fa579844f4ba1aabc1c3009d31d11', 'request_id': 'req-1ae67b6f-80e6-4e68-b182-738101861894'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:36.888263 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-1ae67b6f-80e6-4e68-b182-738101861894 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd54fa579844f4ba1aabc1c3009d31d11', 'request_id': 'req-1ae67b6f-80e6-4e68-b182-738101861894'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:36.901561 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-1ae67b6f-80e6-4e68-b182-738101861894 admin admin] The token cache is being invalidate because project d54fa579844f4ba1aabc1c3009d31d11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:17:36.901849 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-1ae67b6f-80e6-4e68-b182-738101861894 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:36.902470 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-1ae67b6f-80e6-4e68-b182-738101861894 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:36.903968 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1193/2393] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:17:36 2022] DELETE /identity/v3/projects/d54fa579844f4ba1aabc1c3009d31d11 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:17:36.908930 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561596fa-fcae-416e-859e-7e29ba7ed960 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:36.909213 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561596fa-fcae-416e-859e-7e29ba7ed960 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:36.909484 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561596fa-fcae-416e-859e-7e29ba7ed960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:36.929822 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-561596fa-fcae-416e-859e-7e29ba7ed960 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:17:36.930977 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-561596fa-fcae-416e-859e-7e29ba7ed960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:36.964755 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:17:36.990522 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1201/2394] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:17:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:17:36.998416 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec532bb5-cc00-44de-98ff-c2301bdb3f7f None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:36.998704 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec532bb5-cc00-44de-98ff-c2301bdb3f7f None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:36.998977 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec532bb5-cc00-44de-98ff-c2301bdb3f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:37.018711 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-ec532bb5-cc00-44de-98ff-c2301bdb3f7f None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:17:37.020035 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-ec532bb5-cc00-44de-98ff-c2301bdb3f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:37.027834 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1194/2395] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:17:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:17:37.031842 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561596fa-fcae-416e-859e-7e29ba7ed960 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:37.041966 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561596fa-fcae-416e-859e-7e29ba7ed960 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:37.046791 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce3f24-095c-424c-9310-8758d5e16940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:37.047420 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce3f24-095c-424c-9310-8758d5e16940 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:37.047705 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce3f24-095c-424c-9310-8758d5e16940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:37.048246 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce3f24-095c-424c-9310-8758d5e16940 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:37.048893 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdce3f24-095c-424c-9310-8758d5e16940 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:17:37.049197 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdce3f24-095c-424c-9310-8758d5e16940 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:37.051383 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdce3f24-095c-424c-9310-8758d5e16940 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:37.062772 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1202/2396] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:17:37.067255 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec532bb5-cc00-44de-98ff-c2301bdb3f7f None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:37.076364 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec532bb5-cc00-44de-98ff-c2301bdb3f7f None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:37.081005 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ee7ff-3b29-4c8e-91f8-625f4167d3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:37.081594 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ee7ff-3b29-4c8e-91f8-625f4167d3e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:37.081901 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ee7ff-3b29-4c8e-91f8-625f4167d3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:37.082174 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ee7ff-3b29-4c8e-91f8-625f4167d3e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:37.082854 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ee7ff-3b29-4c8e-91f8-625f4167d3e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:37.084783 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ee7ff-3b29-4c8e-91f8-625f4167d3e0 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:37.111513 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1195/2397] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:17:37 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:17:37.115618 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce3f24-095c-424c-9310-8758d5e16940 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:37.125331 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce3f24-095c-424c-9310-8758d5e16940 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:37.181120 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29076207-e9cb-475a-a1ca-6b724c2a700f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:37.181727 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29076207-e9cb-475a-a1ca-6b724c2a700f admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:37.181915 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29076207-e9cb-475a-a1ca-6b724c2a700f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:37.182189 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29076207-e9cb-475a-a1ca-6b724c2a700f admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:37.182824 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29076207-e9cb-475a-a1ca-6b724c2a700f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:37.184770 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29076207-e9cb-475a-a1ca-6b724c2a700f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:37.206488 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1203/2398] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:17:37 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:17:37.211908 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ee7ff-3b29-4c8e-91f8-625f4167d3e0 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:37.222077 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ee7ff-3b29-4c8e-91f8-625f4167d3e0 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:37.226617 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b443a3-9abc-4514-9152-a69d5329175b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:37.227236 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b443a3-9abc-4514-9152-a69d5329175b admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:37.227515 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b443a3-9abc-4514-9152-a69d5329175b admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:37.227788 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b443a3-9abc-4514-9152-a69d5329175b admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:37.228426 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b443a3-9abc-4514-9152-a69d5329175b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:37.230010 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b443a3-9abc-4514-9152-a69d5329175b admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:37.239882 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1196/2399] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:17:37.243506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29076207-e9cb-475a-a1ca-6b724c2a700f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:37.253295 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29076207-e9cb-475a-a1ca-6b724c2a700f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:37.258473 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1179e5d1-0851-42ce-a8a8-ac2efd4d1999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:37.259078 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179e5d1-0851-42ce-a8a8-ac2efd4d1999 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:37.259435 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179e5d1-0851-42ce-a8a8-ac2efd4d1999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:37.259708 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179e5d1-0851-42ce-a8a8-ac2efd4d1999 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:37.260344 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1179e5d1-0851-42ce-a8a8-ac2efd4d1999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:37.261976 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1179e5d1-0851-42ce-a8a8-ac2efd4d1999 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:37.272842 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1204/2400] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:17:37.276701 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b443a3-9abc-4514-9152-a69d5329175b admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:37.286463 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b443a3-9abc-4514-9152-a69d5329175b admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:37.291091 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22acde5-94e1-43dd-aa4a-6eae199884cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:37.291737 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22acde5-94e1-43dd-aa4a-6eae199884cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:37.292016 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22acde5-94e1-43dd-aa4a-6eae199884cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:37.292290 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22acde5-94e1-43dd-aa4a-6eae199884cc admin admin] PATH_INFO: `/v3/projects/17e1654782ac4100b0353d561496db51/users/cd005dcc25944dfc9aeda1eb1468a0ea/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:37.294450 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:17:37.314779 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22acde5-94e1-43dd-aa4a-6eae199884cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=17e1654782ac4100b0353d561496db51, user_id=cd005dcc25944dfc9aeda1eb1468a0ea, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:37.318186 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22acde5-94e1-43dd-aa4a-6eae199884cc admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:37.331011 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1197/2401] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:17:37 2022] PUT /identity/v3/projects/17e1654782ac4100b0353d561496db51/users/cd005dcc25944dfc9aeda1eb1468a0ea/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:17:37.336196 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d842eb-11fc-4fa1-b4bb-11be15f9b0bb None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:37.336484 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d842eb-11fc-4fa1-b4bb-11be15f9b0bb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:37.336773 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d842eb-11fc-4fa1-b4bb-11be15f9b0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:37.355080 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-42d842eb-11fc-4fa1-b4bb-11be15f9b0bb None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:17:37.356276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-42d842eb-11fc-4fa1-b4bb-11be15f9b0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:37.412152 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1205/2402] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:17:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:17:37.421748 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22acde5-94e1-43dd-aa4a-6eae199884cc admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:37.423808 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-d22acde5-94e1-43dd-aa4a-6eae199884cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:37.479534 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22acde5-94e1-43dd-aa4a-6eae199884cc admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:37.483094 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a979f62-2217-41fa-86a5-57a9b9b38613 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:37.483690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a979f62-2217-41fa-86a5-57a9b9b38613 service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:37.483969 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a979f62-2217-41fa-86a5-57a9b9b38613 service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:37.484242 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a979f62-2217-41fa-86a5-57a9b9b38613 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:37.493500 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a979f62-2217-41fa-86a5-57a9b9b38613 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:37.494953 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a979f62-2217-41fa-86a5-57a9b9b38613 service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:37.504839 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1198/2403] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:17:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:17:38.819349 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d842eb-11fc-4fa1-b4bb-11be15f9b0bb None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:38.821412 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-42d842eb-11fc-4fa1-b4bb-11be15f9b0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:38.834618 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d842eb-11fc-4fa1-b4bb-11be15f9b0bb None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:38.849771 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507f483-4010-4292-b406-6fa7968d106d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:38.850491 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507f483-4010-4292-b406-6fa7968d106d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:38.850777 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507f483-4010-4292-b406-6fa7968d106d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:38.851169 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507f483-4010-4292-b406-6fa7968d106d admin admin] PATH_INFO: `/v3/users/e86c98aa51f4412fb688f2c8ffbf0d54` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:38.853191 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2507f483-4010-4292-b406-6fa7968d106d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e86c98aa51f4412fb688f2c8ffbf0d54)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:38.855445 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2507f483-4010-4292-b406-6fa7968d106d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:38.914784 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:17:38.921152 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2507f483-4010-4292-b406-6fa7968d106d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e86c98aa51f4412fb688f2c8ffbf0d54'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:38.931608 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2507f483-4010-4292-b406-6fa7968d106d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e86c98aa51f4412fb688f2c8ffbf0d54'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:38.932459 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2507f483-4010-4292-b406-6fa7968d106d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e86c98aa51f4412fb688f2c8ffbf0d54'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:38.942111 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2507f483-4010-4292-b406-6fa7968d106d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e86c98aa51f4412fb688f2c8ffbf0d54'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:38.963295 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2507f483-4010-4292-b406-6fa7968d106d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e86c98aa51f4412fb688f2c8ffbf0d54'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:38.973733 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1206/2404] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:17:38 2022] DELETE /identity/v3/users/e86c98aa51f4412fb688f2c8ffbf0d54 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:17:38.978886 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0278841-a977-44ba-a0a7-1a12ccbc8b33 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:38.979717 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0278841-a977-44ba-a0a7-1a12ccbc8b33 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:38.980004 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0278841-a977-44ba-a0a7-1a12ccbc8b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:39.000415 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-f0278841-a977-44ba-a0a7-1a12ccbc8b33 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:17:39.001625 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-f0278841-a977-44ba-a0a7-1a12ccbc8b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:39.110984 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1199/2405] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:17:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:17:39.142637 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507f483-4010-4292-b406-6fa7968d106d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:39.144710 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2507f483-4010-4292-b406-6fa7968d106d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:39.252983 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507f483-4010-4292-b406-6fa7968d106d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:39.270739 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe3d18-48b1-48d2-b99b-7d2f0d9f19e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:39.271444 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe3d18-48b1-48d2-b99b-7d2f0d9f19e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:39.271889 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe3d18-48b1-48d2-b99b-7d2f0d9f19e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:39.272348 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe3d18-48b1-48d2-b99b-7d2f0d9f19e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:39.282586 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfe3d18-48b1-48d2-b99b-7d2f0d9f19e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:39.284278 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfe3d18-48b1-48d2-b99b-7d2f0d9f19e9 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:39.294421 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1207/2406] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:17:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:17:39.589206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0278841-a977-44ba-a0a7-1a12ccbc8b33 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:39.590774 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-f0278841-a977-44ba-a0a7-1a12ccbc8b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:39.611868 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0278841-a977-44ba-a0a7-1a12ccbc8b33 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:39.619238 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad6468-f587-4231-9d54-e3682a3137e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:39.620002 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad6468-f587-4231-9d54-e3682a3137e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:39.620346 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad6468-f587-4231-9d54-e3682a3137e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:39.620684 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad6468-f587-4231-9d54-e3682a3137e6 admin admin] PATH_INFO: `/v3/projects/28f27dc791d6469cb0432709d7242c81` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:39.622085 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad6468-f587-4231-9d54-e3682a3137e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28f27dc791d6469cb0432709d7242c81)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:39.624818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad6468-f587-4231-9d54-e3682a3137e6 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:39.681711 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-a2ad6468-f587-4231-9d54-e3682a3137e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28f27dc791d6469cb0432709d7242c81', 'request_id': 'req-a2ad6468-f587-4231-9d54-e3682a3137e6'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:39.688723 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-a2ad6468-f587-4231-9d54-e3682a3137e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28f27dc791d6469cb0432709d7242c81', 'request_id': 'req-a2ad6468-f587-4231-9d54-e3682a3137e6'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:39.699127 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-a2ad6468-f587-4231-9d54-e3682a3137e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28f27dc791d6469cb0432709d7242c81', 'request_id': 'req-a2ad6468-f587-4231-9d54-e3682a3137e6'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:39.714283 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-a2ad6468-f587-4231-9d54-e3682a3137e6 admin admin] The token cache is being invalidate because project 28f27dc791d6469cb0432709d7242c81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:17:39.714844 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-a2ad6468-f587-4231-9d54-e3682a3137e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:39.715674 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-a2ad6468-f587-4231-9d54-e3682a3137e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:39.717410 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1200/2407] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:17:39 2022] DELETE /identity/v3/projects/28f27dc791d6469cb0432709d7242c81 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:17:39.722471 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd798df-995c-40ff-9904-d286c25cc717 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:39.722761 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd798df-995c-40ff-9904-d286c25cc717 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:39.723041 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd798df-995c-40ff-9904-d286c25cc717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:39.744624 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8bd798df-995c-40ff-9904-d286c25cc717 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:17:39.746128 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8bd798df-995c-40ff-9904-d286c25cc717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:39.821948 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1208/2408] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:17:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:17:39.831474 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ad8c3-edcf-477c-8efd-7e7980351d07 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:39.832025 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ad8c3-edcf-477c-8efd-7e7980351d07 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:39.832534 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ad8c3-edcf-477c-8efd-7e7980351d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:39.860141 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-311ad8c3-edcf-477c-8efd-7e7980351d07 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:17:39.861580 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-311ad8c3-edcf-477c-8efd-7e7980351d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:39.873855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1201/2409] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:17:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:17:39.878468 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd798df-995c-40ff-9904-d286c25cc717 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:39.893640 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd798df-995c-40ff-9904-d286c25cc717 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:39.899961 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff56f11-5ae6-48b5-9039-97df80f11417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:39.900748 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff56f11-5ae6-48b5-9039-97df80f11417 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:39.901496 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff56f11-5ae6-48b5-9039-97df80f11417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:39.901855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff56f11-5ae6-48b5-9039-97df80f11417 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:39.902629 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff56f11-5ae6-48b5-9039-97df80f11417 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:17:39.904244 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff56f11-5ae6-48b5-9039-97df80f11417 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:39.906382 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff56f11-5ae6-48b5-9039-97df80f11417 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:39.929123 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1209/2410] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:17:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:17:39.929773 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ad8c3-edcf-477c-8efd-7e7980351d07 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:39.935101 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:17:39.942559 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ad8c3-edcf-477c-8efd-7e7980351d07 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:39.951759 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a544a-29df-476f-9fc5-a6d518fa3b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:39.952615 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a544a-29df-476f-9fc5-a6d518fa3b6c admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:39.953080 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a544a-29df-476f-9fc5-a6d518fa3b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:39.953357 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a544a-29df-476f-9fc5-a6d518fa3b6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:39.954002 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a544a-29df-476f-9fc5-a6d518fa3b6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:39.955990 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a544a-29df-476f-9fc5-a6d518fa3b6c admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:39.985599 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1202/2411] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:17:39 2022] POST /identity/v3/projects => generated 389 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:17:39.990307 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff56f11-5ae6-48b5-9039-97df80f11417 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:40.005964 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff56f11-5ae6-48b5-9039-97df80f11417 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:40.057884 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:17:40.074487 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431155a1-b8fa-43c9-ab66-065a297c3ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:40.075149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431155a1-b8fa-43c9-ab66-065a297c3ce0 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:40.076046 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431155a1-b8fa-43c9-ab66-065a297c3ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:40.076332 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431155a1-b8fa-43c9-ab66-065a297c3ce0 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:40.076973 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431155a1-b8fa-43c9-ab66-065a297c3ce0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:40.078918 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431155a1-b8fa-43c9-ab66-065a297c3ce0 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:40.104563 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1210/2412] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:17:39 2022] POST /identity/v3/users => generated 295 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:17:40.108810 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a544a-29df-476f-9fc5-a6d518fa3b6c admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:40.120168 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a544a-29df-476f-9fc5-a6d518fa3b6c admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:40.125298 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071c589-c15f-4d36-a3a2-7c396d31f846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:40.125996 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071c589-c15f-4d36-a3a2-7c396d31f846 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:40.126352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071c589-c15f-4d36-a3a2-7c396d31f846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:40.126697 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071c589-c15f-4d36-a3a2-7c396d31f846 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:40.127472 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8071c589-c15f-4d36-a3a2-7c396d31f846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:40.129226 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8071c589-c15f-4d36-a3a2-7c396d31f846 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:40.139879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1203/2413] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:17:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:17:40.143849 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431155a1-b8fa-43c9-ab66-065a297c3ce0 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:40.157500 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431155a1-b8fa-43c9-ab66-065a297c3ce0 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:40.166096 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3b91e-8f6f-4bbf-a6bb-8b3e34e157b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:40.166946 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3b91e-8f6f-4bbf-a6bb-8b3e34e157b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:40.167408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3b91e-8f6f-4bbf-a6bb-8b3e34e157b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:40.167834 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3b91e-8f6f-4bbf-a6bb-8b3e34e157b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:40.168658 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3b91e-8f6f-4bbf-a6bb-8b3e34e157b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:40.170421 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3b91e-8f6f-4bbf-a6bb-8b3e34e157b6 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:40.188084 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071c589-c15f-4d36-a3a2-7c396d31f846 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:40.191248 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1211/2414] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:17:40 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:17:40.199593 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071c589-c15f-4d36-a3a2-7c396d31f846 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:40.205673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2f631-8760-4e83-a370-231a34757d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:40.206362 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2f631-8760-4e83-a370-231a34757d01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:40.206878 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2f631-8760-4e83-a370-231a34757d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:40.207526 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2f631-8760-4e83-a370-231a34757d01 admin admin] PATH_INFO: `/v3/projects/08bd230517f84406a4cf1936bf516a6c/users/b18115092b7f44d68dd14fdb32e9acbe/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:40.224976 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2f631-8760-4e83-a370-231a34757d01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08bd230517f84406a4cf1936bf516a6c, user_id=b18115092b7f44d68dd14fdb32e9acbe, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:40.227887 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2f631-8760-4e83-a370-231a34757d01 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:40.244624 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1204/2415] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:17:40 2022] PUT /identity/v3/projects/08bd230517f84406a4cf1936bf516a6c/users/b18115092b7f44d68dd14fdb32e9acbe/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:17:40.255421 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7d80b-0a7e-4bf2-b349-f59abe6477bb None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:40.256094 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7d80b-0a7e-4bf2-b349-f59abe6477bb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:40.256375 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7d80b-0a7e-4bf2-b349-f59abe6477bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:40.289876 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-31d7d80b-0a7e-4bf2-b349-f59abe6477bb None None] MFA Rules not processed for user `b18115092b7f44d68dd14fdb32e9acbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:17:40.291155 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-31d7d80b-0a7e-4bf2-b349-f59abe6477bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:40.364309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1212/2416] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:17:40 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:17:40.375221 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2f631-8760-4e83-a370-231a34757d01 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:40.377556 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-1cd2f631-8760-4e83-a370-231a34757d01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:40.448675 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2f631-8760-4e83-a370-231a34757d01 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:40.452882 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9463c8f3-039b-49f5-81e5-87e2bbad3252 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:40.453510 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463c8f3-039b-49f5-81e5-87e2bbad3252 service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:40.453790 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463c8f3-039b-49f5-81e5-87e2bbad3252 service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:40.454059 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463c8f3-039b-49f5-81e5-87e2bbad3252 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:40.464482 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9463c8f3-039b-49f5-81e5-87e2bbad3252 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:40.466153 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9463c8f3-039b-49f5-81e5-87e2bbad3252 service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:40.478670 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1205/2417] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:17:40 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:17:41.624255 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae0d21-9783-4dc3-a1bf-e9eaac96829c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:41.624747 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae0d21-9783-4dc3-a1bf-e9eaac96829c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:41.624844 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae0d21-9783-4dc3-a1bf-e9eaac96829c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:41.654988 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-e2ae0d21-9783-4dc3-a1bf-e9eaac96829c None None] MFA Rules not processed for user `cd005dcc25944dfc9aeda1eb1468a0ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:17:41.656277 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e2ae0d21-9783-4dc3-a1bf-e9eaac96829c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:41.733521 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1213/2418] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:17:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:17:41.737765 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9463c8f3-039b-49f5-81e5-87e2bbad3252 service cinder] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:41.740553 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-9463c8f3-039b-49f5-81e5-87e2bbad3252 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:41.823947 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9463c8f3-039b-49f5-81e5-87e2bbad3252 service cinder] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:41.828839 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905003e6-f7aa-4099-bcba-5c5c53f11e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:41.829460 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905003e6-f7aa-4099-bcba-5c5c53f11e5e admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:41.829879 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905003e6-f7aa-4099-bcba-5c5c53f11e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:41.830160 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905003e6-f7aa-4099-bcba-5c5c53f11e5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:41.830840 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905003e6-f7aa-4099-bcba-5c5c53f11e5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:41.833494 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905003e6-f7aa-4099-bcba-5c5c53f11e5e admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:41.861983 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1206/2419] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:17:41 2022] POST /identity/v3/projects => generated 379 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:17:41.867359 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae0d21-9783-4dc3-a1bf-e9eaac96829c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:41.881316 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae0d21-9783-4dc3-a1bf-e9eaac96829c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:41.931523 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:17:41.950665 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa233d7-ad59-4dd0-8ef6-bb6720bfe073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:41.951223 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa233d7-ad59-4dd0-8ef6-bb6720bfe073 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:41.951505 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa233d7-ad59-4dd0-8ef6-bb6720bfe073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:41.951775 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa233d7-ad59-4dd0-8ef6-bb6720bfe073 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:41.952415 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa233d7-ad59-4dd0-8ef6-bb6720bfe073 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:41.955855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa233d7-ad59-4dd0-8ef6-bb6720bfe073 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:42.012589 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1214/2420] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:17:41 2022] POST /identity/v3/users => generated 296 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:17:42.015613 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905003e6-f7aa-4099-bcba-5c5c53f11e5e admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:42.027575 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905003e6-f7aa-4099-bcba-5c5c53f11e5e admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:42.033912 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2958ab3-7bd8-42fb-92e8-3e4d0180f66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:42.034644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2958ab3-7bd8-42fb-92e8-3e4d0180f66d admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:42.035793 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2958ab3-7bd8-42fb-92e8-3e4d0180f66d admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:42.036141 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2958ab3-7bd8-42fb-92e8-3e4d0180f66d admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:42.036877 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2958ab3-7bd8-42fb-92e8-3e4d0180f66d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:42.038474 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2958ab3-7bd8-42fb-92e8-3e4d0180f66d admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:42.057190 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1207/2421] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:17:42 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:17:42.061286 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa233d7-ad59-4dd0-8ef6-bb6720bfe073 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:42.072326 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa233d7-ad59-4dd0-8ef6-bb6720bfe073 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:42.077563 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989d2ba-f25a-4d5e-a4f9-0d19e9ae6ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:42.078321 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989d2ba-f25a-4d5e-a4f9-0d19e9ae6ec5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:42.078696 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989d2ba-f25a-4d5e-a4f9-0d19e9ae6ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:42.079041 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989d2ba-f25a-4d5e-a4f9-0d19e9ae6ec5 admin admin] PATH_INFO: `/v3/projects/ad322d9f495a4b33bd84411d7f42df3e/users/9eef02a3b7b642e78d13fbb9d6f036c1/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:42.098275 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989d2ba-f25a-4d5e-a4f9-0d19e9ae6ec5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad322d9f495a4b33bd84411d7f42df3e, user_id=9eef02a3b7b642e78d13fbb9d6f036c1, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:42.101317 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989d2ba-f25a-4d5e-a4f9-0d19e9ae6ec5 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:42.117597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1215/2422] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:17:42 2022] PUT /identity/v3/projects/ad322d9f495a4b33bd84411d7f42df3e/users/9eef02a3b7b642e78d13fbb9d6f036c1/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:17:42.121535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2958ab3-7bd8-42fb-92e8-3e4d0180f66d admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:42.135363 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2958ab3-7bd8-42fb-92e8-3e4d0180f66d admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:42.164239 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:17:42.205821 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46d9aa-c373-4bfd-b6f7-031fdef0c8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:42.206510 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46d9aa-c373-4bfd-b6f7-031fdef0c8db admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:42.207464 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46d9aa-c373-4bfd-b6f7-031fdef0c8db admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:42.207812 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46d9aa-c373-4bfd-b6f7-031fdef0c8db admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:42.208524 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f46d9aa-c373-4bfd-b6f7-031fdef0c8db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:42.210231 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f46d9aa-c373-4bfd-b6f7-031fdef0c8db admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:42.220952 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1208/2423] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:17:42 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:17:42.225190 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989d2ba-f25a-4d5e-a4f9-0d19e9ae6ec5 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:42.235788 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989d2ba-f25a-4d5e-a4f9-0d19e9ae6ec5 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:42.240494 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5e728-b183-44f8-97c5-551d7e7efd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:42.241150 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5e728-b183-44f8-97c5-551d7e7efd8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:42.241426 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5e728-b183-44f8-97c5-551d7e7efd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:42.241693 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5e728-b183-44f8-97c5-551d7e7efd8a admin admin] PATH_INFO: `/v3/projects/ad322d9f495a4b33bd84411d7f42df3e/users/9eef02a3b7b642e78d13fbb9d6f036c1/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:42.243645 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d5e728-b183-44f8-97c5-551d7e7efd8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad322d9f495a4b33bd84411d7f42df3e, user_id=9eef02a3b7b642e78d13fbb9d6f036c1, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:42.246268 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d5e728-b183-44f8-97c5-551d7e7efd8a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:42.258084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1216/2424] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:17:42 2022] PUT /identity/v3/projects/ad322d9f495a4b33bd84411d7f42df3e/users/9eef02a3b7b642e78d13fbb9d6f036c1/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:17:47.152848 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c8b33-24df-4f9b-8242-35071c0ecc65 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:47.153344 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c8b33-24df-4f9b-8242-35071c0ecc65 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:47.153844 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c8b33-24df-4f9b-8242-35071c0ecc65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:47.203235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-8c5c8b33-24df-4f9b-8242-35071c0ecc65 None None] MFA Rules not processed for user `9eef02a3b7b642e78d13fbb9d6f036c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:17:47.209733 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-8c5c8b33-24df-4f9b-8242-35071c0ecc65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:47.294530 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1209/2425] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:17:47 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:17:47.308574 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5e728-b183-44f8-97c5-551d7e7efd8a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:47.310443 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f3d5e728-b183-44f8-97c5-551d7e7efd8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:47.408197 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5e728-b183-44f8-97c5-551d7e7efd8a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:47.415253 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b8b24-562f-4e73-8ae7-27442774798d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:47.415799 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b8b24-562f-4e73-8ae7-27442774798d service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:47.416074 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b8b24-562f-4e73-8ae7-27442774798d service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:47.416346 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b8b24-562f-4e73-8ae7-27442774798d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:47.477625 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4b8b24-562f-4e73-8ae7-27442774798d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:47.479365 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4b8b24-562f-4e73-8ae7-27442774798d service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:47.489095 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1217/2426] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:17:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:17:56.860472 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c8b33-24df-4f9b-8242-35071c0ecc65 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:56.861869 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-8c5c8b33-24df-4f9b-8242-35071c0ecc65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:56.956572 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c8b33-24df-4f9b-8242-35071c0ecc65 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:56.961997 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89b8dd-2771-4ba3-b333-00236ed9dc72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:56.962417 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89b8dd-2771-4ba3-b333-00236ed9dc72 service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:56.962834 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89b8dd-2771-4ba3-b333-00236ed9dc72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:56.963151 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89b8dd-2771-4ba3-b333-00236ed9dc72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:56.965065 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-dd89b8dd-2771-4ba3-b333-00236ed9dc72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:56.975656 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd89b8dd-2771-4ba3-b333-00236ed9dc72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:56.977101 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd89b8dd-2771-4ba3-b333-00236ed9dc72 service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:56.985846 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1210/2427] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:17:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:17:57.060109 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b8b24-562f-4e73-8ae7-27442774798d service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:57.080276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b8b24-562f-4e73-8ae7-27442774798d service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:57.083789 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ff66b-f17b-42f9-9f91-dee9262bd6f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:57.084404 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ff66b-f17b-42f9-9f91-dee9262bd6f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:57.084687 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ff66b-f17b-42f9-9f91-dee9262bd6f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:57.084946 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ff66b-f17b-42f9-9f91-dee9262bd6f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:57.086756 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a99ff66b-f17b-42f9-9f91-dee9262bd6f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:57.118377 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:17:57.123772 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89b8dd-2771-4ba3-b333-00236ed9dc72 service neutron] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:57.145419 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:17:57.152962 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ff66b-f17b-42f9-9f91-dee9262bd6f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:57.154595 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ff66b-f17b-42f9-9f91-dee9262bd6f3 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:57.179016 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1218/2428] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:17:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:17:57.215357 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89b8dd-2771-4ba3-b333-00236ed9dc72 service neutron] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:57.220048 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72444949-e78b-4d6f-b170-c191744475d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:57.220679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72444949-e78b-4d6f-b170-c191744475d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:57.220988 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72444949-e78b-4d6f-b170-c191744475d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:57.221263 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72444949-e78b-4d6f-b170-c191744475d8 admin admin] PATH_INFO: `/v3/users/b18115092b7f44d68dd14fdb32e9acbe` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:57.222924 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72444949-e78b-4d6f-b170-c191744475d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b18115092b7f44d68dd14fdb32e9acbe)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:57.224871 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72444949-e78b-4d6f-b170-c191744475d8 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:57.276979 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-72444949-e78b-4d6f-b170-c191744475d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b18115092b7f44d68dd14fdb32e9acbe'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:57.284875 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-72444949-e78b-4d6f-b170-c191744475d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b18115092b7f44d68dd14fdb32e9acbe'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:57.285536 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-72444949-e78b-4d6f-b170-c191744475d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b18115092b7f44d68dd14fdb32e9acbe'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:57.286143 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-72444949-e78b-4d6f-b170-c191744475d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b18115092b7f44d68dd14fdb32e9acbe'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:57.294845 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-72444949-e78b-4d6f-b170-c191744475d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b18115092b7f44d68dd14fdb32e9acbe'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:57.315216 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1211/2429] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:17:57 2022] DELETE /identity/v3/users/b18115092b7f44d68dd14fdb32e9acbe => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:17:57.319715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b789af4-1436-4cac-ad8d-2ef612b01bbc None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:57.320026 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b789af4-1436-4cac-ad8d-2ef612b01bbc None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:57.320299 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b789af4-1436-4cac-ad8d-2ef612b01bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:57.365887 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-4b789af4-1436-4cac-ad8d-2ef612b01bbc None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:17:57.367144 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4b789af4-1436-4cac-ad8d-2ef612b01bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:57.443231 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1219/2430] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:17:57.454037 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72444949-e78b-4d6f-b170-c191744475d8 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:57.455769 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-72444949-e78b-4d6f-b170-c191744475d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:57.544742 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72444949-e78b-4d6f-b170-c191744475d8 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:57.548667 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d68192-ec29-49ec-8e34-67f481cf67cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:57.549302 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d68192-ec29-49ec-8e34-67f481cf67cd service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:57.550807 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d68192-ec29-49ec-8e34-67f481cf67cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:57.551117 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d68192-ec29-49ec-8e34-67f481cf67cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:57.561952 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d68192-ec29-49ec-8e34-67f481cf67cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:57.563802 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d68192-ec29-49ec-8e34-67f481cf67cd service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:57.576432 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1212/2431] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:17:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:17:57.864006 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b789af4-1436-4cac-ad8d-2ef612b01bbc None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:57.865450 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4b789af4-1436-4cac-ad8d-2ef612b01bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:57.877765 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b789af4-1436-4cac-ad8d-2ef612b01bbc None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:57.882652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba4852-2a93-44bb-91d2-1c8128010d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:57.883872 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba4852-2a93-44bb-91d2-1c8128010d19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:57.884434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba4852-2a93-44bb-91d2-1c8128010d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:57.884940 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba4852-2a93-44bb-91d2-1c8128010d19 admin admin] PATH_INFO: `/v3/projects/08bd230517f84406a4cf1936bf516a6c` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:57.886834 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbba4852-2a93-44bb-91d2-1c8128010d19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08bd230517f84406a4cf1936bf516a6c)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:57.889795 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbba4852-2a93-44bb-91d2-1c8128010d19 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:57.939140 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cbba4852-2a93-44bb-91d2-1c8128010d19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08bd230517f84406a4cf1936bf516a6c', 'request_id': 'req-cbba4852-2a93-44bb-91d2-1c8128010d19'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:57.945466 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cbba4852-2a93-44bb-91d2-1c8128010d19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08bd230517f84406a4cf1936bf516a6c', 'request_id': 'req-cbba4852-2a93-44bb-91d2-1c8128010d19'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:57.952074 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cbba4852-2a93-44bb-91d2-1c8128010d19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08bd230517f84406a4cf1936bf516a6c', 'request_id': 'req-cbba4852-2a93-44bb-91d2-1c8128010d19'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:57.967086 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cbba4852-2a93-44bb-91d2-1c8128010d19 admin admin] The token cache is being invalidate because project 08bd230517f84406a4cf1936bf516a6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:17:57.967413 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cbba4852-2a93-44bb-91d2-1c8128010d19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:57.968097 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cbba4852-2a93-44bb-91d2-1c8128010d19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:17:57.969658 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1220/2432] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:17:57 2022] DELETE /identity/v3/projects/08bd230517f84406a4cf1936bf516a6c => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:17:57.974464 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25954229-0eee-4823-9d60-abf974bfa8af None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:57.974942 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25954229-0eee-4823-9d60-abf974bfa8af None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:57.975075 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25954229-0eee-4823-9d60-abf974bfa8af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:58.004518 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:17:58.037352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-25954229-0eee-4823-9d60-abf974bfa8af None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:17:58.038469 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-25954229-0eee-4823-9d60-abf974bfa8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:58.110360 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1213/2433] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:17:58.117344 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29987f1-0781-4e06-8188-1c476e3da7b6 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:58.117880 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29987f1-0781-4e06-8188-1c476e3da7b6 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:58.118081 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29987f1-0781-4e06-8188-1c476e3da7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:58.140103 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b29987f1-0781-4e06-8188-1c476e3da7b6 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:17:58.141407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b29987f1-0781-4e06-8188-1c476e3da7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:58.150547 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1221/2434] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:17:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:17:58.155912 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25954229-0eee-4823-9d60-abf974bfa8af None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:58.168288 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25954229-0eee-4823-9d60-abf974bfa8af None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:58.170444 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29987f1-0781-4e06-8188-1c476e3da7b6 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:58.173187 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b29987f1-0781-4e06-8188-1c476e3da7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:58.174327 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824071b0-239e-4ead-a3d2-240eb37e9559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:58.174987 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824071b0-239e-4ead-a3d2-240eb37e9559 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:58.175337 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824071b0-239e-4ead-a3d2-240eb37e9559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:58.175614 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824071b0-239e-4ead-a3d2-240eb37e9559 admin admin] PATH_INFO: `/v3/domains` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:58.176365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824071b0-239e-4ead-a3d2-240eb37e9559 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81520) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:17:58.176563 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824071b0-239e-4ead-a3d2-240eb37e9559 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:58.178152 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824071b0-239e-4ead-a3d2-240eb37e9559 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:58.197841 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1214/2435] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:17:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 13 13:17:58.201906 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824071b0-239e-4ead-a3d2-240eb37e9559 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:58.209311 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:17:58.213701 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824071b0-239e-4ead-a3d2-240eb37e9559 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:58.218762 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ff695-ddda-43ee-99ea-5754e5eaccf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:58.219521 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ff695-ddda-43ee-99ea-5754e5eaccf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:58.219804 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ff695-ddda-43ee-99ea-5754e5eaccf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:58.220075 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ff695-ddda-43ee-99ea-5754e5eaccf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:58.220701 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ff695-ddda-43ee-99ea-5754e5eaccf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:58.222652 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ff695-ddda-43ee-99ea-5754e5eaccf9 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:58.257257 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1215/2436] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:17:58 2022] POST /identity/v3/projects => generated 399 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:17:58.258254 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ff695-ddda-43ee-99ea-5754e5eaccf9 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:58.283178 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29987f1-0781-4e06-8188-1c476e3da7b6 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:58.322407 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ff695-ddda-43ee-99ea-5754e5eaccf9 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:58.327498 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67d438-787d-4487-9ad1-b37c539dcdc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:58.328102 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67d438-787d-4487-9ad1-b37c539dcdc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:58.328378 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67d438-787d-4487-9ad1-b37c539dcdc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:58.328651 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67d438-787d-4487-9ad1-b37c539dcdc7 admin admin] PATH_INFO: `/v3/users` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:58.329282 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff67d438-787d-4487-9ad1-b37c539dcdc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:58.331250 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff67d438-787d-4487-9ad1-b37c539dcdc7 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:58.345027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf1f40-c8dc-4156-b9be-3864b967e8f6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a971cdb7f9d4e6692c6326c103d7303', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:58.345928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf1f40-c8dc-4156-b9be-3864b967e8f6 service placement] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:58.346169 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf1f40-c8dc-4156-b9be-3864b967e8f6 service placement] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:58.346411 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf1f40-c8dc-4156-b9be-3864b967e8f6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:58.348571 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a7cf1f40-c8dc-4156-b9be-3864b967e8f6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:58.356612 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1216/2437] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:17:58 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:17:58.361074 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67d438-787d-4487-9ad1-b37c539dcdc7 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:58.362780 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cf1f40-c8dc-4156-b9be-3864b967e8f6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:58.364433 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cf1f40-c8dc-4156-b9be-3864b967e8f6 service placement] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:58.373797 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67d438-787d-4487-9ad1-b37c539dcdc7 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:58.375383 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1222/2438] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:17:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 13 13:17:58.382299 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5007c97-81fc-47b7-a0c8-ead5914afecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:58.383188 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5007c97-81fc-47b7-a0c8-ead5914afecb admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:58.383669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5007c97-81fc-47b7-a0c8-ead5914afecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:58.384126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5007c97-81fc-47b7-a0c8-ead5914afecb admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:58.384951 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5007c97-81fc-47b7-a0c8-ead5914afecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:58.387105 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5007c97-81fc-47b7-a0c8-ead5914afecb admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:58.401656 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1217/2439] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:17:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:17:58.402701 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf1f40-c8dc-4156-b9be-3864b967e8f6 service placement] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:58.413477 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf1f40-c8dc-4156-b9be-3864b967e8f6 service placement] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:58.419548 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b14f2-008f-4df7-a821-56febced44a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:58.420434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b14f2-008f-4df7-a821-56febced44a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:58.420861 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b14f2-008f-4df7-a821-56febced44a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:58.427492 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b14f2-008f-4df7-a821-56febced44a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:58.428418 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b14f2-008f-4df7-a821-56febced44a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:58.430082 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b14f2-008f-4df7-a821-56febced44a6 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:58.442689 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1223/2440] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:17:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:17:58.446746 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5007c97-81fc-47b7-a0c8-ead5914afecb admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:58.457320 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5007c97-81fc-47b7-a0c8-ead5914afecb admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:58.463550 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d599d6-bfcf-4781-973f-e71c60c04969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:58.464231 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d599d6-bfcf-4781-973f-e71c60c04969 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:58.464514 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d599d6-bfcf-4781-973f-e71c60c04969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:58.464820 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d599d6-bfcf-4781-973f-e71c60c04969 admin admin] PATH_INFO: `/v3/projects/88b54594d23e4bd499fe9b8b5955f92a/users/b959a0dea1504e0d98191f78615a2486/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:58.483767 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d599d6-bfcf-4781-973f-e71c60c04969 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88b54594d23e4bd499fe9b8b5955f92a, user_id=b959a0dea1504e0d98191f78615a2486, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:58.486846 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d599d6-bfcf-4781-973f-e71c60c04969 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:58.512706 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1218/2441] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:17:58 2022] PUT /identity/v3/projects/88b54594d23e4bd499fe9b8b5955f92a/users/b959a0dea1504e0d98191f78615a2486/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:17:58.522598 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9796434-a166-420a-8ff5-e30067e18edc None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:58.522888 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9796434-a166-420a-8ff5-e30067e18edc None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:58.523202 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9796434-a166-420a-8ff5-e30067e18edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:58.561656 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b9796434-a166-420a-8ff5-e30067e18edc None None] MFA Rules not processed for user `b959a0dea1504e0d98191f78615a2486`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:17:58.562907 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b9796434-a166-420a-8ff5-e30067e18edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:58.633638 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1224/2442] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:17:58 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:17:58.645013 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d599d6-bfcf-4781-973f-e71c60c04969 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:17:58.646762 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-a8d599d6-bfcf-4781-973f-e71c60c04969 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:17:58.668571 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:17:58.709237 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d599d6-bfcf-4781-973f-e71c60c04969 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:17:58.713170 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2578c1-3f7d-4d96-93f7-c53180863b37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:17:58.713762 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2578c1-3f7d-4d96-93f7-c53180863b37 service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:17:58.714193 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2578c1-3f7d-4d96-93f7-c53180863b37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:17:58.714339 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2578c1-3f7d-4d96-93f7-c53180863b37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:17:58.724777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2578c1-3f7d-4d96-93f7-c53180863b37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:17:58.726242 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2578c1-3f7d-4d96-93f7-c53180863b37 service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:17:58.736466 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1219/2443] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:17:58 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:18:06.820700 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9796434-a166-420a-8ff5-e30067e18edc None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:18:06.841145 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:18:06.888031 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9796434-a166-420a-8ff5-e30067e18edc None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:18:06.892495 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c8203-42e5-4e79-8094-f80c7f6010d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:18:06.893119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c8203-42e5-4e79-8094-f80c7f6010d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:18:06.893397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c8203-42e5-4e79-8094-f80c7f6010d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:18:06.893665 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c8203-42e5-4e79-8094-f80c7f6010d8 admin admin] PATH_INFO: `/v3/users/b959a0dea1504e0d98191f78615a2486` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:18:06.894784 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c8203-42e5-4e79-8094-f80c7f6010d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b959a0dea1504e0d98191f78615a2486)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:18:06.896759 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c8203-42e5-4e79-8094-f80c7f6010d8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:18:06.945870 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-509c8203-42e5-4e79-8094-f80c7f6010d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b959a0dea1504e0d98191f78615a2486'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:18:06.953293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-509c8203-42e5-4e79-8094-f80c7f6010d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b959a0dea1504e0d98191f78615a2486'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:18:06.954060 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-509c8203-42e5-4e79-8094-f80c7f6010d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b959a0dea1504e0d98191f78615a2486'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:18:06.962489 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-509c8203-42e5-4e79-8094-f80c7f6010d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b959a0dea1504e0d98191f78615a2486'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:18:06.976689 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-509c8203-42e5-4e79-8094-f80c7f6010d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b959a0dea1504e0d98191f78615a2486'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:18:06.982874 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1225/2444] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:18:06 2022] DELETE /identity/v3/users/b959a0dea1504e0d98191f78615a2486 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:18:06.987166 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c276d-c031-460f-95f9-f2e326157f82 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:18:06.988019 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c276d-c031-460f-95f9-f2e326157f82 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:18:06.989000 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c276d-c031-460f-95f9-f2e326157f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:18:07.012666 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-5f9c276d-c031-460f-95f9-f2e326157f82 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:18:07.014332 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-5f9c276d-c031-460f-95f9-f2e326157f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:18:07.073509 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1220/2445] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:18:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:18:07.083280 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c8203-42e5-4e79-8094-f80c7f6010d8 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:18:07.084688 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-509c8203-42e5-4e79-8094-f80c7f6010d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:18:07.140611 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c8203-42e5-4e79-8094-f80c7f6010d8 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:18:07.144224 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b329a11-2ff8-45ca-915a-49420170613b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:18:07.144818 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b329a11-2ff8-45ca-915a-49420170613b service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:18:07.145095 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b329a11-2ff8-45ca-915a-49420170613b service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:18:07.145364 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b329a11-2ff8-45ca-915a-49420170613b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:18:07.154487 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b329a11-2ff8-45ca-915a-49420170613b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:18:07.155831 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b329a11-2ff8-45ca-915a-49420170613b service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:18:07.165919 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1226/2446] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:18:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:18:07.447798 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c276d-c031-460f-95f9-f2e326157f82 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:18:07.449366 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-5f9c276d-c031-460f-95f9-f2e326157f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:18:07.463494 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c276d-c031-460f-95f9-f2e326157f82 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:18:07.468376 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55cc795-6354-49ee-b56e-40655cf68fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:18:07.469036 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55cc795-6354-49ee-b56e-40655cf68fa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:18:07.469319 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55cc795-6354-49ee-b56e-40655cf68fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:18:07.469592 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55cc795-6354-49ee-b56e-40655cf68fa9 admin admin] PATH_INFO: `/v3/projects/88b54594d23e4bd499fe9b8b5955f92a` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:18:07.470778 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55cc795-6354-49ee-b56e-40655cf68fa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88b54594d23e4bd499fe9b8b5955f92a)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:18:07.473508 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55cc795-6354-49ee-b56e-40655cf68fa9 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:18:07.526914 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-a55cc795-6354-49ee-b56e-40655cf68fa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88b54594d23e4bd499fe9b8b5955f92a', 'request_id': 'req-a55cc795-6354-49ee-b56e-40655cf68fa9'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:18:07.533920 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-a55cc795-6354-49ee-b56e-40655cf68fa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88b54594d23e4bd499fe9b8b5955f92a', 'request_id': 'req-a55cc795-6354-49ee-b56e-40655cf68fa9'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:18:07.541144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-a55cc795-6354-49ee-b56e-40655cf68fa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88b54594d23e4bd499fe9b8b5955f92a', 'request_id': 'req-a55cc795-6354-49ee-b56e-40655cf68fa9'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:18:07.554985 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-a55cc795-6354-49ee-b56e-40655cf68fa9 admin admin] The token cache is being invalidate because project 88b54594d23e4bd499fe9b8b5955f92a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:18:07.555285 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-a55cc795-6354-49ee-b56e-40655cf68fa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:18:07.556028 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-a55cc795-6354-49ee-b56e-40655cf68fa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:18:07.557774 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1221/2447] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:18:07 2022] DELETE /identity/v3/projects/88b54594d23e4bd499fe9b8b5955f92a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:18:08.159784 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f5b15-7bde-44b8-ae9b-e040b0e1f02f None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:18:08.160274 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f5b15-7bde-44b8-ae9b-e040b0e1f02f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:18:08.160392 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f5b15-7bde-44b8-ae9b-e040b0e1f02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:18:08.179819 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-619f5b15-7bde-44b8-ae9b-e040b0e1f02f None None] MFA Rules not processed for user `cd005dcc25944dfc9aeda1eb1468a0ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:18:08.181036 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-619f5b15-7bde-44b8-ae9b-e040b0e1f02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:18:08.212349 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:18:08.236623 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1227/2448] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:18:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:18:12.446579 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d2a1d-cf69-4da9-8038-8d96ca29fffc None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:18:12.447140 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d2a1d-cf69-4da9-8038-8d96ca29fffc None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:18:12.447296 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d2a1d-cf69-4da9-8038-8d96ca29fffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:18:12.479236 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-866d2a1d-cf69-4da9-8038-8d96ca29fffc None None] MFA Rules not processed for user `2cd12070eb954d6880f925957184a288`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:18:12.480456 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-866d2a1d-cf69-4da9-8038-8d96ca29fffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:18:12.530056 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:18:12.548235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1222/2449] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:18:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:18:18.229962 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=8 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:18:22.541710 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=5 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:18:25.698464 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f5b15-7bde-44b8-ae9b-e040b0e1f02f None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:18:25.699880 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-619f5b15-7bde-44b8-ae9b-e040b0e1f02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:18:25.774828 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f5b15-7bde-44b8-ae9b-e040b0e1f02f None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:18:25.778362 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2b8d0-ff26-491f-979d-14d5591d0482 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:18:25.778954 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2b8d0-ff26-491f-979d-14d5591d0482 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:18:25.779257 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2b8d0-ff26-491f-979d-14d5591d0482 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:18:25.779530 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2b8d0-ff26-491f-979d-14d5591d0482 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:18:25.781218 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fac2b8d0-ff26-491f-979d-14d5591d0482 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:18:25.841807 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac2b8d0-ff26-491f-979d-14d5591d0482 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:18:25.843257 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac2b8d0-ff26-491f-979d-14d5591d0482 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:18:25.852506 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1228/2450] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:18:25 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:18:35.848332 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=59 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:19:11.401629 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b3ec8-770d-4262-af7c-7a6f22536b51 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:19:11.402152 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b3ec8-770d-4262-af7c-7a6f22536b51 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:19:11.402267 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b3ec8-770d-4262-af7c-7a6f22536b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:19:11.420825 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-dc4b3ec8-770d-4262-af7c-7a6f22536b51 None None] MFA Rules not processed for user `2cd12070eb954d6880f925957184a288`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:19:11.422262 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-dc4b3ec8-770d-4262-af7c-7a6f22536b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:19:11.429059 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1223/2451] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:19:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:19:21.416039 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:19:32.468521 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2b8d0-ff26-491f-979d-14d5591d0482 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:19:32.469956 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fac2b8d0-ff26-491f-979d-14d5591d0482 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:19:32.543782 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2b8d0-ff26-491f-979d-14d5591d0482 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:19:32.549456 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c51021e-e561-4840-bff4-91fc854a3e1a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a971cdb7f9d4e6692c6326c103d7303', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:19:32.550312 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51021e-e561-4840-bff4-91fc854a3e1a service placement] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:19:32.551184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51021e-e561-4840-bff4-91fc854a3e1a service placement] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:19:32.551184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51021e-e561-4840-bff4-91fc854a3e1a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:19:32.553650 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4c51021e-e561-4840-bff4-91fc854a3e1a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:19:32.565375 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c51021e-e561-4840-bff4-91fc854a3e1a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:19:32.566938 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c51021e-e561-4840-bff4-91fc854a3e1a service placement] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:19:32.576703 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1229/2452] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:19:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:19:35.604258 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b3ec8-770d-4262-af7c-7a6f22536b51 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:19:35.616994 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b3ec8-770d-4262-af7c-7a6f22536b51 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:19:35.622007 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e8aef-47b5-483b-83e3-c942fd4ffc98 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a971cdb7f9d4e6692c6326c103d7303', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:19:35.622647 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e8aef-47b5-483b-83e3-c942fd4ffc98 service placement] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:19:35.622920 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e8aef-47b5-483b-83e3-c942fd4ffc98 service placement] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:19:35.623374 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e8aef-47b5-483b-83e3-c942fd4ffc98 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:19:35.625288 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-1e9e8aef-47b5-483b-83e3-c942fd4ffc98 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:19:35.637233 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e8aef-47b5-483b-83e3-c942fd4ffc98 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:19:35.638901 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e8aef-47b5-483b-83e3-c942fd4ffc98 service placement] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:19:35.650634 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1224/2453] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:19:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:19:35.657724 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c51021e-e561-4840-bff4-91fc854a3e1a service placement] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:19:35.666083 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4c51021e-e561-4840-bff4-91fc854a3e1a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:19:35.725288 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c51021e-e561-4840-bff4-91fc854a3e1a service placement] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:19:35.729750 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b78ee-b24d-4181-bcfc-ac6161f54503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:19:35.730362 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b78ee-b24d-4181-bcfc-ac6161f54503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:19:35.730637 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b78ee-b24d-4181-bcfc-ac6161f54503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:19:35.730903 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b78ee-b24d-4181-bcfc-ac6161f54503 admin admin] PATH_INFO: `/v3/users/6723d3c8aa144452b46564649cf38ee3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:19:35.732033 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2b78ee-b24d-4181-bcfc-ac6161f54503 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6723d3c8aa144452b46564649cf38ee3)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:19:35.733925 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2b78ee-b24d-4181-bcfc-ac6161f54503 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:19:35.777596 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ff2b78ee-b24d-4181-bcfc-ac6161f54503 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6723d3c8aa144452b46564649cf38ee3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:19:35.784208 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ff2b78ee-b24d-4181-bcfc-ac6161f54503 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6723d3c8aa144452b46564649cf38ee3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:19:35.784859 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ff2b78ee-b24d-4181-bcfc-ac6161f54503 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6723d3c8aa144452b46564649cf38ee3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:19:35.791963 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ff2b78ee-b24d-4181-bcfc-ac6161f54503 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6723d3c8aa144452b46564649cf38ee3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:19:35.804638 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ff2b78ee-b24d-4181-bcfc-ac6161f54503 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6723d3c8aa144452b46564649cf38ee3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:19:35.810586 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1230/2454] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:19:35 2022] DELETE /identity/v3/users/6723d3c8aa144452b46564649cf38ee3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:19:35.983006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e8aef-47b5-483b-83e3-c942fd4ffc98 service placement] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:19:35.995620 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e8aef-47b5-483b-83e3-c942fd4ffc98 service placement] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:19:36.052091 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da867c4-cdca-4d67-938e-eee335fd9786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:19:36.052711 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da867c4-cdca-4d67-938e-eee335fd9786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:19:36.052997 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da867c4-cdca-4d67-938e-eee335fd9786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:19:36.053270 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da867c4-cdca-4d67-938e-eee335fd9786 admin admin] PATH_INFO: `/v3/projects/aff81c38c17b4e82864854ceac6587df` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:19:36.054369 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da867c4-cdca-4d67-938e-eee335fd9786 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aff81c38c17b4e82864854ceac6587df)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:19:36.056304 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da867c4-cdca-4d67-938e-eee335fd9786 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:19:36.100636 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-2da867c4-cdca-4d67-938e-eee335fd9786 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aff81c38c17b4e82864854ceac6587df', 'request_id': 'req-2da867c4-cdca-4d67-938e-eee335fd9786'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:19:36.106268 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-2da867c4-cdca-4d67-938e-eee335fd9786 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aff81c38c17b4e82864854ceac6587df', 'request_id': 'req-2da867c4-cdca-4d67-938e-eee335fd9786'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:19:36.113199 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-2da867c4-cdca-4d67-938e-eee335fd9786 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aff81c38c17b4e82864854ceac6587df', 'request_id': 'req-2da867c4-cdca-4d67-938e-eee335fd9786'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:19:36.125157 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-2da867c4-cdca-4d67-938e-eee335fd9786 admin admin] The token cache is being invalidate because project aff81c38c17b4e82864854ceac6587df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:19:36.125443 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-2da867c4-cdca-4d67-938e-eee335fd9786 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:19:36.126089 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-2da867c4-cdca-4d67-938e-eee335fd9786 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:19:36.127590 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1225/2455] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:19:35 2022] DELETE /identity/v3/projects/aff81c38c17b4e82864854ceac6587df => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:19:36.132287 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b78ee-b24d-4181-bcfc-ac6161f54503 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:19:36.133696 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ff2b78ee-b24d-4181-bcfc-ac6161f54503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:19:36.147125 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b78ee-b24d-4181-bcfc-ac6161f54503 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:19:36.151639 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:19:36.204374 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bafdd00-9e21-44db-b7a0-767eb7bcd6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:19:36.204998 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bafdd00-9e21-44db-b7a0-767eb7bcd6bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:19:36.205282 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bafdd00-9e21-44db-b7a0-767eb7bcd6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:19:36.205556 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bafdd00-9e21-44db-b7a0-767eb7bcd6bf admin admin] PATH_INFO: `/v3/users/64d0f55920a54472828632465d053d66` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:19:36.206684 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bafdd00-9e21-44db-b7a0-767eb7bcd6bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=64d0f55920a54472828632465d053d66)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:19:36.209228 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bafdd00-9e21-44db-b7a0-767eb7bcd6bf admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:19:36.253293 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6bafdd00-9e21-44db-b7a0-767eb7bcd6bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64d0f55920a54472828632465d053d66'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:19:36.260531 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6bafdd00-9e21-44db-b7a0-767eb7bcd6bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64d0f55920a54472828632465d053d66'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:19:36.261179 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6bafdd00-9e21-44db-b7a0-767eb7bcd6bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64d0f55920a54472828632465d053d66'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:19:36.268009 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6bafdd00-9e21-44db-b7a0-767eb7bcd6bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64d0f55920a54472828632465d053d66'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:19:36.280376 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6bafdd00-9e21-44db-b7a0-767eb7bcd6bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64d0f55920a54472828632465d053d66'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:19:36.286770 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1231/2456] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:19:36 2022] DELETE /identity/v3/users/64d0f55920a54472828632465d053d66 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:19:36.466072 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da867c4-cdca-4d67-938e-eee335fd9786 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:19:36.467831 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-2da867c4-cdca-4d67-938e-eee335fd9786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:19:36.484885 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da867c4-cdca-4d67-938e-eee335fd9786 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:19:36.541414 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fdd84-1266-4167-ad3e-3ab54c867440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:19:36.541967 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fdd84-1266-4167-ad3e-3ab54c867440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:19:36.542244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fdd84-1266-4167-ad3e-3ab54c867440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:19:36.542516 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fdd84-1266-4167-ad3e-3ab54c867440 admin admin] PATH_INFO: `/v3/projects/cdb20a29152744f580ec016b985892d5` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:19:36.543647 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512fdd84-1266-4167-ad3e-3ab54c867440 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdb20a29152744f580ec016b985892d5)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:19:36.545541 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512fdd84-1266-4167-ad3e-3ab54c867440 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:19:36.566206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:19:36.592079 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-512fdd84-1266-4167-ad3e-3ab54c867440 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdb20a29152744f580ec016b985892d5', 'request_id': 'req-512fdd84-1266-4167-ad3e-3ab54c867440'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:19:36.598044 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-512fdd84-1266-4167-ad3e-3ab54c867440 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdb20a29152744f580ec016b985892d5', 'request_id': 'req-512fdd84-1266-4167-ad3e-3ab54c867440'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:19:36.604301 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-512fdd84-1266-4167-ad3e-3ab54c867440 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdb20a29152744f580ec016b985892d5', 'request_id': 'req-512fdd84-1266-4167-ad3e-3ab54c867440'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:19:36.616761 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-512fdd84-1266-4167-ad3e-3ab54c867440 admin admin] The token cache is being invalidate because project cdb20a29152744f580ec016b985892d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:19:36.617047 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-512fdd84-1266-4167-ad3e-3ab54c867440 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:19:36.617663 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-512fdd84-1266-4167-ad3e-3ab54c867440 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:19:36.619227 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1226/2457] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:19:36 2022] DELETE /identity/v3/projects/cdb20a29152744f580ec016b985892d5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:19:46.282628 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:19:46.613170 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:19:57.981017 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bafdd00-9e21-44db-b7a0-767eb7bcd6bf admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:19:57.982623 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6bafdd00-9e21-44db-b7a0-767eb7bcd6bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:19:58.059089 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bafdd00-9e21-44db-b7a0-767eb7bcd6bf admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:19:58.105987 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13beb4-58b6-438b-bab3-efd642e8f4b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:19:58.106585 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13beb4-58b6-438b-bab3-efd642e8f4b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:19:58.106866 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13beb4-58b6-438b-bab3-efd642e8f4b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:19:58.107172 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13beb4-58b6-438b-bab3-efd642e8f4b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:19:58.108887 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fb13beb4-58b6-438b-bab3-efd642e8f4b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:19:58.176392 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb13beb4-58b6-438b-bab3-efd642e8f4b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:19:58.178234 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb13beb4-58b6-438b-bab3-efd642e8f4b6 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:19:58.189164 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1232/2458] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:19:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:20:04.913976 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fdd84-1266-4167-ad3e-3ab54c867440 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:20:04.915443 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-512fdd84-1266-4167-ad3e-3ab54c867440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:20:04.928677 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fdd84-1266-4167-ad3e-3ab54c867440 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:20:04.932618 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bff48d-be3b-4bdc-8b47-527a43dd31c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:20:04.933323 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bff48d-be3b-4bdc-8b47-527a43dd31c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:20:04.933700 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bff48d-be3b-4bdc-8b47-527a43dd31c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:20:04.934072 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bff48d-be3b-4bdc-8b47-527a43dd31c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:20:04.935922 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-31bff48d-be3b-4bdc-8b47-527a43dd31c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:20:04.949558 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bff48d-be3b-4bdc-8b47-527a43dd31c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:20:04.950985 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bff48d-be3b-4bdc-8b47-527a43dd31c7 service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:20:04.962191 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1227/2459] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:20:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 13 13:20:08.184256 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=83 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:20:11.835664 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c440229-0ab7-48c8-ae6d-160a2ab9dec4 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:20:11.836281 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c440229-0ab7-48c8-ae6d-160a2ab9dec4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:20:11.836281 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c440229-0ab7-48c8-ae6d-160a2ab9dec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:20:11.855428 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-4c440229-0ab7-48c8-ae6d-160a2ab9dec4 None None] MFA Rules not processed for user `2cd12070eb954d6880f925957184a288`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:20:11.856627 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4c440229-0ab7-48c8-ae6d-160a2ab9dec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:20:11.908693 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1233/2460] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:20:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:20:14.956910 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=6 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:20:15.953224 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bff48d-be3b-4bdc-8b47-527a43dd31c7 service cinder] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:20:15.954653 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-31bff48d-be3b-4bdc-8b47-527a43dd31c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:20:16.012226 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bff48d-be3b-4bdc-8b47-527a43dd31c7 service cinder] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:20:16.015824 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59b2c8-beef-4715-aab8-b04060fc07ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:20:16.016417 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59b2c8-beef-4715-aab8-b04060fc07ad service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:20:16.016691 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59b2c8-beef-4715-aab8-b04060fc07ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:20:16.016958 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59b2c8-beef-4715-aab8-b04060fc07ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:20:16.018699 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-2d59b2c8-beef-4715-aab8-b04060fc07ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:20:16.028365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d59b2c8-beef-4715-aab8-b04060fc07ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:20:16.029798 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d59b2c8-beef-4715-aab8-b04060fc07ad service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:20:16.038115 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1228/2461] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:20:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:20:21.902182 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=25 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:20:26.034061 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=26 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:20:28.442366 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf1607-5c4d-4225-a3e9-662fbf506e98 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:20:28.442836 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf1607-5c4d-4225-a3e9-662fbf506e98 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:20:28.442955 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf1607-5c4d-4225-a3e9-662fbf506e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:20:28.461413 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-e0cf1607-5c4d-4225-a3e9-662fbf506e98 None None] MFA Rules not processed for user `cd005dcc25944dfc9aeda1eb1468a0ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:20:28.462603 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e0cf1607-5c4d-4225-a3e9-662fbf506e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:20:28.522018 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1234/2462] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:20:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:20:29.263735 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59b2c8-beef-4715-aab8-b04060fc07ad service neutron] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:20:29.265175 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-2d59b2c8-beef-4715-aab8-b04060fc07ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:20:29.340014 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59b2c8-beef-4715-aab8-b04060fc07ad service neutron] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:20:29.344918 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb78fd1-daa0-4e31-b5ef-d38f1388dfb9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a971cdb7f9d4e6692c6326c103d7303', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:20:29.345513 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb78fd1-daa0-4e31-b5ef-d38f1388dfb9 service placement] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:20:29.345790 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb78fd1-daa0-4e31-b5ef-d38f1388dfb9 service placement] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:20:29.346060 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb78fd1-daa0-4e31-b5ef-d38f1388dfb9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:20:29.347752 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-7cb78fd1-daa0-4e31-b5ef-d38f1388dfb9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:20:29.359094 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb78fd1-daa0-4e31-b5ef-d38f1388dfb9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:20:29.360643 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb78fd1-daa0-4e31-b5ef-d38f1388dfb9 service placement] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:20:29.370205 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1229/2463] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:20:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:20:31.664921 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf1607-5c4d-4225-a3e9-662fbf506e98 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:20:31.667710 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e0cf1607-5c4d-4225-a3e9-662fbf506e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:20:31.686074 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf1607-5c4d-4225-a3e9-662fbf506e98 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:20:31.692430 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56bade-5a36-41dd-a6df-db2039429a5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:20:31.693012 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56bade-5a36-41dd-a6df-db2039429a5f service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:20:31.693292 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56bade-5a36-41dd-a6df-db2039429a5f service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:20:31.693558 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56bade-5a36-41dd-a6df-db2039429a5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:20:31.695349 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5a56bade-5a36-41dd-a6df-db2039429a5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:20:31.709566 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56bade-5a36-41dd-a6df-db2039429a5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:20:31.711296 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56bade-5a36-41dd-a6df-db2039429a5f service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:20:31.722614 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1235/2464] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:20:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:20:39.366137 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=32 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:20:41.291395 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb78fd1-daa0-4e31-b5ef-d38f1388dfb9 service placement] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:20:41.292838 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-7cb78fd1-daa0-4e31-b5ef-d38f1388dfb9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:20:41.308223 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb78fd1-daa0-4e31-b5ef-d38f1388dfb9 service placement] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:20:41.313648 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3875639-0464-4bd1-ae69-1270abeea6e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:20:41.314250 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3875639-0464-4bd1-ae69-1270abeea6e7 service nova] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:20:41.314523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3875639-0464-4bd1-ae69-1270abeea6e7 service nova] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:20:41.314790 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3875639-0464-4bd1-ae69-1270abeea6e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:20:41.317041 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-f3875639-0464-4bd1-ae69-1270abeea6e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:20:41.329310 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3875639-0464-4bd1-ae69-1270abeea6e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:20:41.330890 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3875639-0464-4bd1-ae69-1270abeea6e7 service nova] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:20:41.342145 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1230/2465] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:20:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:20:41.717489 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=31 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:20:51.337803 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=6 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:18.231390 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56bade-5a36-41dd-a6df-db2039429a5f service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:18.232888 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5a56bade-5a36-41dd-a6df-db2039429a5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:18.305344 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56bade-5a36-41dd-a6df-db2039429a5f service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:18.308625 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c8d4c-7726-4369-9857-74ba1aa7de57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:18.309212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c8d4c-7726-4369-9857-74ba1aa7de57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:18.309491 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c8d4c-7726-4369-9857-74ba1aa7de57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:18.309770 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c8d4c-7726-4369-9857-74ba1aa7de57 admin admin] PATH_INFO: `/v3/users/2cd12070eb954d6880f925957184a288` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:18.310884 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c8d4c-7726-4369-9857-74ba1aa7de57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cd12070eb954d6880f925957184a288)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:18.312850 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c8d4c-7726-4369-9857-74ba1aa7de57 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:18.358437 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-211c8d4c-7726-4369-9857-74ba1aa7de57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cd12070eb954d6880f925957184a288'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:18.365610 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-211c8d4c-7726-4369-9857-74ba1aa7de57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cd12070eb954d6880f925957184a288'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:18.366269 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-211c8d4c-7726-4369-9857-74ba1aa7de57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cd12070eb954d6880f925957184a288'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:18.373963 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-211c8d4c-7726-4369-9857-74ba1aa7de57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cd12070eb954d6880f925957184a288'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:18.386434 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-211c8d4c-7726-4369-9857-74ba1aa7de57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cd12070eb954d6880f925957184a288'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:18.392274 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1236/2466] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:21:18 2022] DELETE /identity/v3/users/2cd12070eb954d6880f925957184a288 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:18.565547 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3875639-0464-4bd1-ae69-1270abeea6e7 service nova] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:18.578510 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3875639-0464-4bd1-ae69-1270abeea6e7 service nova] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:18.635591 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fdf09c-3b34-4f61-9783-f3d3bfc9a941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:18.636142 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fdf09c-3b34-4f61-9783-f3d3bfc9a941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:18.636420 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fdf09c-3b34-4f61-9783-f3d3bfc9a941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:18.636689 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fdf09c-3b34-4f61-9783-f3d3bfc9a941 admin admin] PATH_INFO: `/v3/projects/da546353194342439f9bc3f98f6737e0` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:18.637780 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fdf09c-3b34-4f61-9783-f3d3bfc9a941 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da546353194342439f9bc3f98f6737e0)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:18.639698 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fdf09c-3b34-4f61-9783-f3d3bfc9a941 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:18.685089 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c6fdf09c-3b34-4f61-9783-f3d3bfc9a941 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da546353194342439f9bc3f98f6737e0', 'request_id': 'req-c6fdf09c-3b34-4f61-9783-f3d3bfc9a941'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:18.690786 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c6fdf09c-3b34-4f61-9783-f3d3bfc9a941 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da546353194342439f9bc3f98f6737e0', 'request_id': 'req-c6fdf09c-3b34-4f61-9783-f3d3bfc9a941'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:18.697097 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c6fdf09c-3b34-4f61-9783-f3d3bfc9a941 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da546353194342439f9bc3f98f6737e0', 'request_id': 'req-c6fdf09c-3b34-4f61-9783-f3d3bfc9a941'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:18.709342 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c6fdf09c-3b34-4f61-9783-f3d3bfc9a941 admin admin] The token cache is being invalidate because project da546353194342439f9bc3f98f6737e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:21:18.709632 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c6fdf09c-3b34-4f61-9783-f3d3bfc9a941 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:18.710266 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c6fdf09c-3b34-4f61-9783-f3d3bfc9a941 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:18.711776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1231/2467] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:21:18 2022] DELETE /identity/v3/projects/da546353194342439f9bc3f98f6737e0 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:18.716629 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6988f-72e7-43ef-89e9-ab7cf382deb1 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:18.716924 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6988f-72e7-43ef-89e9-ab7cf382deb1 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:18.717202 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6988f-72e7-43ef-89e9-ab7cf382deb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:18.736998 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-00d6988f-72e7-43ef-89e9-ab7cf382deb1 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:21:18.738224 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-00d6988f-72e7-43ef-89e9-ab7cf382deb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:18.796412 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1237/2468] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:21:18.805126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385131c6-2b71-470c-95f0-2e9bdf964fd1 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:18.805433 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385131c6-2b71-470c-95f0-2e9bdf964fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:18.805709 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385131c6-2b71-470c-95f0-2e9bdf964fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:18.824885 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-385131c6-2b71-470c-95f0-2e9bdf964fd1 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:21:18.826420 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-385131c6-2b71-470c-95f0-2e9bdf964fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:18.834164 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1232/2469] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:21:18.838259 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6988f-72e7-43ef-89e9-ab7cf382deb1 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:18.847784 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6988f-72e7-43ef-89e9-ab7cf382deb1 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:18.852883 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689e6a2-5716-4428-880d-ed91ec7c0432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:18.853473 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689e6a2-5716-4428-880d-ed91ec7c0432 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:18.853755 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689e6a2-5716-4428-880d-ed91ec7c0432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:18.854030 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689e6a2-5716-4428-880d-ed91ec7c0432 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:18.854672 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7689e6a2-5716-4428-880d-ed91ec7c0432 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:21:18.854973 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7689e6a2-5716-4428-880d-ed91ec7c0432 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:18.856586 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7689e6a2-5716-4428-880d-ed91ec7c0432 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:18.868308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1238/2470] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:21:18.872390 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385131c6-2b71-470c-95f0-2e9bdf964fd1 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:18.881392 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385131c6-2b71-470c-95f0-2e9bdf964fd1 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:18.885974 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175007b-0858-473a-bd33-bee2a9aadaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:18.886527 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175007b-0858-473a-bd33-bee2a9aadaaf admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:18.886800 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175007b-0858-473a-bd33-bee2a9aadaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:18.887080 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175007b-0858-473a-bd33-bee2a9aadaaf admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:18.887711 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e175007b-0858-473a-bd33-bee2a9aadaaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:18.889591 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e175007b-0858-473a-bd33-bee2a9aadaaf admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:18.928606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1233/2471] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:21:18 2022] POST /identity/v3/projects => generated 401 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:21:18.932778 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689e6a2-5716-4428-880d-ed91ec7c0432 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:18.942607 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689e6a2-5716-4428-880d-ed91ec7c0432 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:18.946118 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:19.000367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a361870-0c37-42f0-8c86-bcbfb5f1e0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:19.000961 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a361870-0c37-42f0-8c86-bcbfb5f1e0de admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:19.001242 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a361870-0c37-42f0-8c86-bcbfb5f1e0de admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:19.001513 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a361870-0c37-42f0-8c86-bcbfb5f1e0de admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:19.002136 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a361870-0c37-42f0-8c86-bcbfb5f1e0de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:19.004079 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a361870-0c37-42f0-8c86-bcbfb5f1e0de admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:19.025522 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1239/2472] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:21:18 2022] POST /identity/v3/users => generated 301 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:21:19.029868 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175007b-0858-473a-bd33-bee2a9aadaaf admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:19.040937 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175007b-0858-473a-bd33-bee2a9aadaaf admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:19.045542 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c11e4a-9e15-4d0f-a7da-a621585f3d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:19.046137 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c11e4a-9e15-4d0f-a7da-a621585f3d65 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:19.046417 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c11e4a-9e15-4d0f-a7da-a621585f3d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:19.046950 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c11e4a-9e15-4d0f-a7da-a621585f3d65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:19.047635 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c11e4a-9e15-4d0f-a7da-a621585f3d65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:19.049225 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c11e4a-9e15-4d0f-a7da-a621585f3d65 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:19.058970 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1234/2473] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:21:19.062744 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a361870-0c37-42f0-8c86-bcbfb5f1e0de admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:19.072572 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a361870-0c37-42f0-8c86-bcbfb5f1e0de admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:19.077126 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91557f-6fa6-45c7-a55b-3555ff370e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:19.077711 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91557f-6fa6-45c7-a55b-3555ff370e8e admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:19.077991 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91557f-6fa6-45c7-a55b-3555ff370e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:19.078260 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91557f-6fa6-45c7-a55b-3555ff370e8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:19.078894 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a91557f-6fa6-45c7-a55b-3555ff370e8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:19.080547 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a91557f-6fa6-45c7-a55b-3555ff370e8e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:19.091569 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1240/2474] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:21:19.095461 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c11e4a-9e15-4d0f-a7da-a621585f3d65 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:19.106173 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c11e4a-9e15-4d0f-a7da-a621585f3d65 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:19.110837 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8996d-9f86-4e11-9e08-62448f42baa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:19.111512 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8996d-9f86-4e11-9e08-62448f42baa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:19.111791 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8996d-9f86-4e11-9e08-62448f42baa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:19.112065 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8996d-9f86-4e11-9e08-62448f42baa8 admin admin] PATH_INFO: `/v3/projects/69a6112a9a8e460ba98751edeace719d/users/acc2f572f31b4a12b434ac9ab823bb0b/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:19.126319 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc8996d-9f86-4e11-9e08-62448f42baa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69a6112a9a8e460ba98751edeace719d, user_id=acc2f572f31b4a12b434ac9ab823bb0b, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:19.128971 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc8996d-9f86-4e11-9e08-62448f42baa8 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:19.141399 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1235/2475] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:21:19 2022] PUT /identity/v3/projects/69a6112a9a8e460ba98751edeace719d/users/acc2f572f31b4a12b434ac9ab823bb0b/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:19.146940 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334623c-fc51-402b-b577-b175333583b9 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:19.147272 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334623c-fc51-402b-b577-b175333583b9 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:19.147568 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334623c-fc51-402b-b577-b175333583b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:19.166258 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-1334623c-fc51-402b-b577-b175333583b9 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:21:19.167495 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1334623c-fc51-402b-b577-b175333583b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:19.224740 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1241/2476] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:21:19.234355 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8996d-9f86-4e11-9e08-62448f42baa8 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:19.236066 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-4fc8996d-9f86-4e11-9e08-62448f42baa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:19.256050 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:19.300546 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8996d-9f86-4e11-9e08-62448f42baa8 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:19.305045 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e0592-1a28-49dc-8f65-414947c6ec84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:19.306283 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e0592-1a28-49dc-8f65-414947c6ec84 service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:19.306283 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e0592-1a28-49dc-8f65-414947c6ec84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:19.306283 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e0592-1a28-49dc-8f65-414947c6ec84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:19.315833 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e0592-1a28-49dc-8f65-414947c6ec84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:19.317313 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e0592-1a28-49dc-8f65-414947c6ec84 service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:19.327195 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1236/2477] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:21:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:21:19.931089 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fa074-9562-45f0-8a60-0086ed372b6e None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:19.931621 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fa074-9562-45f0-8a60-0086ed372b6e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:19.931810 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fa074-9562-45f0-8a60-0086ed372b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:19.959893 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-ba7fa074-9562-45f0-8a60-0086ed372b6e None None] MFA Rules not processed for user `acc2f572f31b4a12b434ac9ab823bb0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:21:19.961360 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ba7fa074-9562-45f0-8a60-0086ed372b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:20.027300 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1242/2478] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:21:19 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:21:20.031971 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e0592-1a28-49dc-8f65-414947c6ec84 service neutron] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:20.041911 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e0592-1a28-49dc-8f65-414947c6ec84 service neutron] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:20.047034 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f896976-dce8-47b7-ad4b-67ee96f60bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:20.047731 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f896976-dce8-47b7-ad4b-67ee96f60bd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:20.048074 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f896976-dce8-47b7-ad4b-67ee96f60bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:20.048431 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f896976-dce8-47b7-ad4b-67ee96f60bd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:20.049125 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f896976-dce8-47b7-ad4b-67ee96f60bd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:20.051635 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f896976-dce8-47b7-ad4b-67ee96f60bd2 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:20.080232 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1237/2479] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:21:20 2022] POST /identity/v3/projects => generated 403 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:21:20.084651 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fa074-9562-45f0-8a60-0086ed372b6e None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:20.095959 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fa074-9562-45f0-8a60-0086ed372b6e None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:20.108243 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:20.157850 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98819981-c5d4-4ab9-ab30-ef71ec423104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:20.158759 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98819981-c5d4-4ab9-ab30-ef71ec423104 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:20.159042 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98819981-c5d4-4ab9-ab30-ef71ec423104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:20.159349 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98819981-c5d4-4ab9-ab30-ef71ec423104 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:20.160007 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98819981-c5d4-4ab9-ab30-ef71ec423104 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:20.162442 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98819981-c5d4-4ab9-ab30-ef71ec423104 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:20.186024 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1243/2480] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:21:20 2022] POST /identity/v3/users => generated 308 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:21:20.190271 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f896976-dce8-47b7-ad4b-67ee96f60bd2 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:20.201696 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f896976-dce8-47b7-ad4b-67ee96f60bd2 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:20.206368 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490bfb3f-686a-4e17-961e-c591bf45ba9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:20.206968 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bfb3f-686a-4e17-961e-c591bf45ba9c admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:20.207287 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bfb3f-686a-4e17-961e-c591bf45ba9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:20.207682 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bfb3f-686a-4e17-961e-c591bf45ba9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:20.208329 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490bfb3f-686a-4e17-961e-c591bf45ba9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:20.209987 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490bfb3f-686a-4e17-961e-c591bf45ba9c admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:20.221238 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1238/2481] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:21:20.226155 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98819981-c5d4-4ab9-ab30-ef71ec423104 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:20.236142 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98819981-c5d4-4ab9-ab30-ef71ec423104 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:20.240882 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27728d-a3ab-4856-a8fc-a05fc9ea1aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:20.241530 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27728d-a3ab-4856-a8fc-a05fc9ea1aba admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:20.241806 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27728d-a3ab-4856-a8fc-a05fc9ea1aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:20.242090 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27728d-a3ab-4856-a8fc-a05fc9ea1aba admin admin] PATH_INFO: `/v3/projects/b35318bb7bba486d9cb661968c103ba3/users/a7d14f1c99d94dfba9b16a427e3816e4/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:20.258112 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d27728d-a3ab-4856-a8fc-a05fc9ea1aba admin admin] RBAC: Authorizing `identity:create_grant(project_id=b35318bb7bba486d9cb661968c103ba3, user_id=a7d14f1c99d94dfba9b16a427e3816e4, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:20.260865 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d27728d-a3ab-4856-a8fc-a05fc9ea1aba admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:20.274418 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1244/2482] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:21:20 2022] PUT /identity/v3/projects/b35318bb7bba486d9cb661968c103ba3/users/a7d14f1c99d94dfba9b16a427e3816e4/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:20.278559 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490bfb3f-686a-4e17-961e-c591bf45ba9c admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:20.288478 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490bfb3f-686a-4e17-961e-c591bf45ba9c admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:20.344064 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e97e474-7046-4f42-b6cf-e1b18bfbba16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:20.344670 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97e474-7046-4f42-b6cf-e1b18bfbba16 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:20.344953 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97e474-7046-4f42-b6cf-e1b18bfbba16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:20.345228 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97e474-7046-4f42-b6cf-e1b18bfbba16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:20.345864 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e97e474-7046-4f42-b6cf-e1b18bfbba16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:20.347502 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e97e474-7046-4f42-b6cf-e1b18bfbba16 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:20.357027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1239/2483] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:21:20.361267 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27728d-a3ab-4856-a8fc-a05fc9ea1aba admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:20.371465 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27728d-a3ab-4856-a8fc-a05fc9ea1aba admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:20.376124 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6a47b-99f7-497a-8187-1983138f198e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:20.376772 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6a47b-99f7-497a-8187-1983138f198e admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:20.377055 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6a47b-99f7-497a-8187-1983138f198e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:20.377331 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6a47b-99f7-497a-8187-1983138f198e admin admin] PATH_INFO: `/v3/projects/b35318bb7bba486d9cb661968c103ba3/users/a7d14f1c99d94dfba9b16a427e3816e4/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:20.379809 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed6a47b-99f7-497a-8187-1983138f198e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b35318bb7bba486d9cb661968c103ba3, user_id=a7d14f1c99d94dfba9b16a427e3816e4, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:20.382425 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed6a47b-99f7-497a-8187-1983138f198e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:20.392409 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1245/2484] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:21:20 2022] PUT /identity/v3/projects/b35318bb7bba486d9cb661968c103ba3/users/a7d14f1c99d94dfba9b16a427e3816e4/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:21.011445 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096b9bc-4b24-4f22-bc73-7f5133f62672 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:21.011659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096b9bc-4b24-4f22-bc73-7f5133f62672 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:21.011929 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096b9bc-4b24-4f22-bc73-7f5133f62672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:21.040037 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-4096b9bc-4b24-4f22-bc73-7f5133f62672 None None] MFA Rules not processed for user `a7d14f1c99d94dfba9b16a427e3816e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:21:21.041295 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-4096b9bc-4b24-4f22-bc73-7f5133f62672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:21.111093 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1240/2485] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:21:21 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:21:22.120302 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6a47b-99f7-497a-8187-1983138f198e admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:22.186033 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6a47b-99f7-497a-8187-1983138f198e admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:22.190849 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84206fe-da17-4e9a-b71e-6e36a38bd5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:22.191511 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84206fe-da17-4e9a-b71e-6e36a38bd5f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:22.191831 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84206fe-da17-4e9a-b71e-6e36a38bd5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:22.192613 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84206fe-da17-4e9a-b71e-6e36a38bd5f6 admin admin] PATH_INFO: `/v3/users/acc2f572f31b4a12b434ac9ab823bb0b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:22.193740 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84206fe-da17-4e9a-b71e-6e36a38bd5f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acc2f572f31b4a12b434ac9ab823bb0b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:22.195717 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84206fe-da17-4e9a-b71e-6e36a38bd5f6 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:22.242231 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a84206fe-da17-4e9a-b71e-6e36a38bd5f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acc2f572f31b4a12b434ac9ab823bb0b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:22.248871 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a84206fe-da17-4e9a-b71e-6e36a38bd5f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acc2f572f31b4a12b434ac9ab823bb0b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:22.249524 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a84206fe-da17-4e9a-b71e-6e36a38bd5f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acc2f572f31b4a12b434ac9ab823bb0b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:22.257311 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a84206fe-da17-4e9a-b71e-6e36a38bd5f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acc2f572f31b4a12b434ac9ab823bb0b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:22.262921 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=85,keystone:INSERT=6,keystone:DELETE=9 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:22.272695 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a84206fe-da17-4e9a-b71e-6e36a38bd5f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acc2f572f31b4a12b434ac9ab823bb0b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:22.279218 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1246/2486] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:21:22 2022] DELETE /identity/v3/users/acc2f572f31b4a12b434ac9ab823bb0b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:22.448847 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096b9bc-4b24-4f22-bc73-7f5133f62672 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:22.459489 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096b9bc-4b24-4f22-bc73-7f5133f62672 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:22.464163 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:22.517776 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16791c-13ca-4f7f-bc25-fde1bc856d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:22.518397 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16791c-13ca-4f7f-bc25-fde1bc856d6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:22.518675 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16791c-13ca-4f7f-bc25-fde1bc856d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:22.518943 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16791c-13ca-4f7f-bc25-fde1bc856d6d admin admin] PATH_INFO: `/v3/projects/69a6112a9a8e460ba98751edeace719d` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:22.520093 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e16791c-13ca-4f7f-bc25-fde1bc856d6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=69a6112a9a8e460ba98751edeace719d)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:22.521991 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e16791c-13ca-4f7f-bc25-fde1bc856d6d admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:22.566496 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-5e16791c-13ca-4f7f-bc25-fde1bc856d6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69a6112a9a8e460ba98751edeace719d', 'request_id': 'req-5e16791c-13ca-4f7f-bc25-fde1bc856d6d'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:22.571948 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-5e16791c-13ca-4f7f-bc25-fde1bc856d6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69a6112a9a8e460ba98751edeace719d', 'request_id': 'req-5e16791c-13ca-4f7f-bc25-fde1bc856d6d'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:22.577862 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-5e16791c-13ca-4f7f-bc25-fde1bc856d6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69a6112a9a8e460ba98751edeace719d', 'request_id': 'req-5e16791c-13ca-4f7f-bc25-fde1bc856d6d'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:22.589531 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-5e16791c-13ca-4f7f-bc25-fde1bc856d6d admin admin] The token cache is being invalidate because project 69a6112a9a8e460ba98751edeace719d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:21:22.589818 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-5e16791c-13ca-4f7f-bc25-fde1bc856d6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:22.590453 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-5e16791c-13ca-4f7f-bc25-fde1bc856d6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:22.591945 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1241/2487] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:21:22 2022] DELETE /identity/v3/projects/69a6112a9a8e460ba98751edeace719d => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:22.596312 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84206fe-da17-4e9a-b71e-6e36a38bd5f6 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:22.597691 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a84206fe-da17-4e9a-b71e-6e36a38bd5f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:22.610704 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84206fe-da17-4e9a-b71e-6e36a38bd5f6 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:22.665141 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cfb310-7a56-4e3f-b76e-239c43cdbf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:22.665688 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cfb310-7a56-4e3f-b76e-239c43cdbf05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:22.665963 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cfb310-7a56-4e3f-b76e-239c43cdbf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:22.666229 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cfb310-7a56-4e3f-b76e-239c43cdbf05 admin admin] PATH_INFO: `/v3/users/a7d14f1c99d94dfba9b16a427e3816e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:22.667356 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cfb310-7a56-4e3f-b76e-239c43cdbf05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7d14f1c99d94dfba9b16a427e3816e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:22.669244 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cfb310-7a56-4e3f-b76e-239c43cdbf05 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:22.718143 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-08cfb310-7a56-4e3f-b76e-239c43cdbf05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7d14f1c99d94dfba9b16a427e3816e4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:22.725018 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-08cfb310-7a56-4e3f-b76e-239c43cdbf05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7d14f1c99d94dfba9b16a427e3816e4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:22.725695 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-08cfb310-7a56-4e3f-b76e-239c43cdbf05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7d14f1c99d94dfba9b16a427e3816e4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:22.732136 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-08cfb310-7a56-4e3f-b76e-239c43cdbf05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7d14f1c99d94dfba9b16a427e3816e4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:22.743649 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-08cfb310-7a56-4e3f-b76e-239c43cdbf05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7d14f1c99d94dfba9b16a427e3816e4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:22.749348 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1247/2488] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:21:22 2022] DELETE /identity/v3/users/a7d14f1c99d94dfba9b16a427e3816e4 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:22.916790 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16791c-13ca-4f7f-bc25-fde1bc856d6d admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:22.918407 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-5e16791c-13ca-4f7f-bc25-fde1bc856d6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:22.932671 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16791c-13ca-4f7f-bc25-fde1bc856d6d admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:22.989615 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab24776-c69c-4e8d-9dae-fd36835e2253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:22.990316 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab24776-c69c-4e8d-9dae-fd36835e2253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:22.990590 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab24776-c69c-4e8d-9dae-fd36835e2253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:22.990858 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab24776-c69c-4e8d-9dae-fd36835e2253 admin admin] PATH_INFO: `/v3/projects/b35318bb7bba486d9cb661968c103ba3` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:22.992013 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab24776-c69c-4e8d-9dae-fd36835e2253 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b35318bb7bba486d9cb661968c103ba3)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:22.994059 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab24776-c69c-4e8d-9dae-fd36835e2253 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:23.033297 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:23.040195 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-6ab24776-c69c-4e8d-9dae-fd36835e2253 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b35318bb7bba486d9cb661968c103ba3', 'request_id': 'req-6ab24776-c69c-4e8d-9dae-fd36835e2253'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:23.046043 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-6ab24776-c69c-4e8d-9dae-fd36835e2253 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b35318bb7bba486d9cb661968c103ba3', 'request_id': 'req-6ab24776-c69c-4e8d-9dae-fd36835e2253'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:23.052855 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-6ab24776-c69c-4e8d-9dae-fd36835e2253 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b35318bb7bba486d9cb661968c103ba3', 'request_id': 'req-6ab24776-c69c-4e8d-9dae-fd36835e2253'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:23.065450 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-6ab24776-c69c-4e8d-9dae-fd36835e2253 admin admin] The token cache is being invalidate because project b35318bb7bba486d9cb661968c103ba3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:21:23.065737 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-6ab24776-c69c-4e8d-9dae-fd36835e2253 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:23.066349 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-6ab24776-c69c-4e8d-9dae-fd36835e2253 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:23.067913 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1242/2489] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:21:22 2022] DELETE /identity/v3/projects/b35318bb7bba486d9cb661968c103ba3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:23.072760 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653a184-5363-40a2-95b4-d962eca3ac34 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:23.073053 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653a184-5363-40a2-95b4-d962eca3ac34 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:23.073327 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653a184-5363-40a2-95b4-d962eca3ac34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:23.093162 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-1653a184-5363-40a2-95b4-d962eca3ac34 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:21:23.094347 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1653a184-5363-40a2-95b4-d962eca3ac34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:23.150508 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1248/2490] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:21:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:21:23.158101 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d329a94-9b0e-4069-ba66-eaac2f43628f None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:23.158396 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d329a94-9b0e-4069-ba66-eaac2f43628f None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:23.158670 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d329a94-9b0e-4069-ba66-eaac2f43628f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:23.179287 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-3d329a94-9b0e-4069-ba66-eaac2f43628f None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:21:23.180431 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-3d329a94-9b0e-4069-ba66-eaac2f43628f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:23.188206 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1243/2491] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:21:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:21:23.192220 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1653a184-5363-40a2-95b4-d962eca3ac34 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:23.201516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1653a184-5363-40a2-95b4-d962eca3ac34 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:23.205986 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e8889-9262-43b7-b2f3-3419e5460e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:23.206575 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e8889-9262-43b7-b2f3-3419e5460e71 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:23.206856 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e8889-9262-43b7-b2f3-3419e5460e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:23.207154 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e8889-9262-43b7-b2f3-3419e5460e71 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:23.207798 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e8889-9262-43b7-b2f3-3419e5460e71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:21:23.208092 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e8889-9262-43b7-b2f3-3419e5460e71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:23.209654 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e8889-9262-43b7-b2f3-3419e5460e71 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:23.211099 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:23.226078 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1249/2492] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:21:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:21:23.227766 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d329a94-9b0e-4069-ba66-eaac2f43628f None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:23.236986 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d329a94-9b0e-4069-ba66-eaac2f43628f None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:23.242148 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e6ba0-33be-4173-bda5-13c282859d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:23.242736 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e6ba0-33be-4173-bda5-13c282859d38 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:23.243009 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e6ba0-33be-4173-bda5-13c282859d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:23.243308 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e6ba0-33be-4173-bda5-13c282859d38 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:23.243935 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76e6ba0-33be-4173-bda5-13c282859d38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:23.245801 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76e6ba0-33be-4173-bda5-13c282859d38 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:23.272244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1244/2493] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:21:23 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:21:23.276435 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e8889-9262-43b7-b2f3-3419e5460e71 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:23.286816 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e8889-9262-43b7-b2f3-3419e5460e71 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:23.340906 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14664ea-981c-4e0f-bdc9-6740d3c34d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:23.341499 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14664ea-981c-4e0f-bdc9-6740d3c34d3a admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:23.341776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14664ea-981c-4e0f-bdc9-6740d3c34d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:23.342047 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14664ea-981c-4e0f-bdc9-6740d3c34d3a admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:23.342668 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14664ea-981c-4e0f-bdc9-6740d3c34d3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:23.344597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14664ea-981c-4e0f-bdc9-6740d3c34d3a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:23.365970 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1250/2494] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:21:23 2022] POST /identity/v3/users => generated 291 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:21:23.370052 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e6ba0-33be-4173-bda5-13c282859d38 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:23.380467 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e6ba0-33be-4173-bda5-13c282859d38 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:23.385224 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d0e78-a571-46c6-9735-f205736b4f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:23.385979 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d0e78-a571-46c6-9735-f205736b4f3b admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:23.386260 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d0e78-a571-46c6-9735-f205736b4f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:23.386531 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d0e78-a571-46c6-9735-f205736b4f3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:23.387213 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4d0e78-a571-46c6-9735-f205736b4f3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:23.388806 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4d0e78-a571-46c6-9735-f205736b4f3b admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:23.400085 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1245/2495] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:21:23.403928 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14664ea-981c-4e0f-bdc9-6740d3c34d3a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:23.414274 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14664ea-981c-4e0f-bdc9-6740d3c34d3a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:23.418850 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb94a93-1617-42cf-b29a-63c717bbd22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:23.419474 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb94a93-1617-42cf-b29a-63c717bbd22e admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:23.419758 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb94a93-1617-42cf-b29a-63c717bbd22e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:23.420033 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb94a93-1617-42cf-b29a-63c717bbd22e admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:23.420672 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb94a93-1617-42cf-b29a-63c717bbd22e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:23.422546 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb94a93-1617-42cf-b29a-63c717bbd22e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:23.433633 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1251/2496] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 13 13:21:23.437556 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d0e78-a571-46c6-9735-f205736b4f3b admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:23.447409 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d0e78-a571-46c6-9735-f205736b4f3b admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:23.452118 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95415776-2087-412c-8ac4-e50d02863a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:23.452768 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95415776-2087-412c-8ac4-e50d02863a6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:23.453046 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95415776-2087-412c-8ac4-e50d02863a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:23.453317 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95415776-2087-412c-8ac4-e50d02863a6b admin admin] PATH_INFO: `/v3/projects/effd0bd0a8894e29918fd3776d71ccad/users/03986507df5245458411daed053e3ca8/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:23.467644 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95415776-2087-412c-8ac4-e50d02863a6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=effd0bd0a8894e29918fd3776d71ccad, user_id=03986507df5245458411daed053e3ca8, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:23.470249 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95415776-2087-412c-8ac4-e50d02863a6b admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:23.483583 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1246/2497] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:21:23 2022] PUT /identity/v3/projects/effd0bd0a8894e29918fd3776d71ccad/users/03986507df5245458411daed053e3ca8/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:23.493887 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58232b3-636a-450b-b1a9-17c2ab2c2821 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:23.494193 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58232b3-636a-450b-b1a9-17c2ab2c2821 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:23.494466 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58232b3-636a-450b-b1a9-17c2ab2c2821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:23.522154 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b58232b3-636a-450b-b1a9-17c2ab2c2821 None None] MFA Rules not processed for user `03986507df5245458411daed053e3ca8`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:21:23.523379 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b58232b3-636a-450b-b1a9-17c2ab2c2821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:23.584419 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1252/2498] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:21:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:21:23.588852 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95415776-2087-412c-8ac4-e50d02863a6b admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:23.650356 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95415776-2087-412c-8ac4-e50d02863a6b admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:23.654998 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e16d8-139e-4871-a6e3-610a430301b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:23.655630 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e16d8-139e-4871-a6e3-610a430301b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:23.655908 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e16d8-139e-4871-a6e3-610a430301b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:23.656176 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e16d8-139e-4871-a6e3-610a430301b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:23.656798 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e16d8-139e-4871-a6e3-610a430301b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:23.658686 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e16d8-139e-4871-a6e3-610a430301b5 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:23.685013 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1247/2499] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:21:23 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:21:23.689005 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58232b3-636a-450b-b1a9-17c2ab2c2821 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:23.699650 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58232b3-636a-450b-b1a9-17c2ab2c2821 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:23.755653 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2281bc0-847c-4124-a3e7-3b128f2d40ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:23.756242 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2281bc0-847c-4124-a3e7-3b128f2d40ff admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:23.756519 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2281bc0-847c-4124-a3e7-3b128f2d40ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:23.756785 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2281bc0-847c-4124-a3e7-3b128f2d40ff admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:23.757411 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2281bc0-847c-4124-a3e7-3b128f2d40ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:23.759333 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2281bc0-847c-4124-a3e7-3b128f2d40ff admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:23.768023 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:23.781030 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1253/2500] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:21:23 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:21:23.785001 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e16d8-139e-4871-a6e3-610a430301b5 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:23.795325 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e16d8-139e-4871-a6e3-610a430301b5 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:23.800164 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b94f40-4b7d-4d06-999d-001ea5ab0bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:23.800756 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b94f40-4b7d-4d06-999d-001ea5ab0bea admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:23.801035 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b94f40-4b7d-4d06-999d-001ea5ab0bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:23.801796 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b94f40-4b7d-4d06-999d-001ea5ab0bea admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:23.802433 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b94f40-4b7d-4d06-999d-001ea5ab0bea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:23.804063 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b94f40-4b7d-4d06-999d-001ea5ab0bea admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:23.814340 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1248/2501] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:21:23.818920 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2281bc0-847c-4124-a3e7-3b128f2d40ff admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:23.828529 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2281bc0-847c-4124-a3e7-3b128f2d40ff admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:23.833466 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c626a80e-6641-45b0-bda3-612eeca0c574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:23.834100 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626a80e-6641-45b0-bda3-612eeca0c574 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:23.834375 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626a80e-6641-45b0-bda3-612eeca0c574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:23.834644 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626a80e-6641-45b0-bda3-612eeca0c574 admin admin] PATH_INFO: `/v3/projects/b899f578308b4498aaab98a78db19875/users/91dbd4e9f7844166b3d4fb42c9442f37/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:23.848783 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c626a80e-6641-45b0-bda3-612eeca0c574 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b899f578308b4498aaab98a78db19875, user_id=91dbd4e9f7844166b3d4fb42c9442f37, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:23.851464 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c626a80e-6641-45b0-bda3-612eeca0c574 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:23.864194 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1254/2502] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:21:23 2022] PUT /identity/v3/projects/b899f578308b4498aaab98a78db19875/users/91dbd4e9f7844166b3d4fb42c9442f37/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:21:23.868046 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b94f40-4b7d-4d06-999d-001ea5ab0bea admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:23.877896 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b94f40-4b7d-4d06-999d-001ea5ab0bea admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:23.916002 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:23.936300 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802267b2-7dc5-4268-bcfe-5bdcc30c3c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:23.936826 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802267b2-7dc5-4268-bcfe-5bdcc30c3c35 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:23.937104 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802267b2-7dc5-4268-bcfe-5bdcc30c3c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:23.937373 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802267b2-7dc5-4268-bcfe-5bdcc30c3c35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:23.938007 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802267b2-7dc5-4268-bcfe-5bdcc30c3c35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:23.939617 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802267b2-7dc5-4268-bcfe-5bdcc30c3c35 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:23.949049 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1249/2503] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:21:23.953324 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c626a80e-6641-45b0-bda3-612eeca0c574 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:23.963316 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c626a80e-6641-45b0-bda3-612eeca0c574 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:23.967808 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82515d8d-9b33-47aa-be5d-12fe9d587fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:23.968442 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82515d8d-9b33-47aa-be5d-12fe9d587fb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:23.968718 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82515d8d-9b33-47aa-be5d-12fe9d587fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:23.968988 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82515d8d-9b33-47aa-be5d-12fe9d587fb9 admin admin] PATH_INFO: `/v3/projects/b899f578308b4498aaab98a78db19875/users/91dbd4e9f7844166b3d4fb42c9442f37/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:23.970873 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82515d8d-9b33-47aa-be5d-12fe9d587fb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b899f578308b4498aaab98a78db19875, user_id=91dbd4e9f7844166b3d4fb42c9442f37, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:23.974547 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82515d8d-9b33-47aa-be5d-12fe9d587fb9 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:23.984654 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1255/2504] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:21:23 2022] PUT /identity/v3/projects/b899f578308b4498aaab98a78db19875/users/91dbd4e9f7844166b3d4fb42c9442f37/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:23.996360 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e42cd-9096-44c2-874f-dc8108559d3a None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:23.996647 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e42cd-9096-44c2-874f-dc8108559d3a None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:23.996946 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e42cd-9096-44c2-874f-dc8108559d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:24.024712 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-589e42cd-9096-44c2-874f-dc8108559d3a None None] MFA Rules not processed for user `91dbd4e9f7844166b3d4fb42c9442f37`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:21:24.025926 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-589e42cd-9096-44c2-874f-dc8108559d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:24.091498 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1250/2505] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:21:23 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:21:24.103150 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82515d8d-9b33-47aa-be5d-12fe9d587fb9 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:24.104865 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-82515d8d-9b33-47aa-be5d-12fe9d587fb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:24.160345 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82515d8d-9b33-47aa-be5d-12fe9d587fb9 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:24.164558 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef6b05-e30c-46ef-a312-807d7a0a802a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:24.165157 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef6b05-e30c-46ef-a312-807d7a0a802a service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:24.165440 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef6b05-e30c-46ef-a312-807d7a0a802a service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:24.165715 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef6b05-e30c-46ef-a312-807d7a0a802a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:24.175146 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef6b05-e30c-46ef-a312-807d7a0a802a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:24.177222 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef6b05-e30c-46ef-a312-807d7a0a802a service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:24.186554 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1256/2506] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:21:24 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:21:24.261047 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e42cd-9096-44c2-874f-dc8108559d3a None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:24.271118 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e42cd-9096-44c2-874f-dc8108559d3a None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:24.274656 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff942b63-3fc0-4784-97af-bc955ccf9ffc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:24.275275 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff942b63-3fc0-4784-97af-bc955ccf9ffc service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:24.275547 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff942b63-3fc0-4784-97af-bc955ccf9ffc service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:24.275814 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff942b63-3fc0-4784-97af-bc955ccf9ffc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:24.330634 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff942b63-3fc0-4784-97af-bc955ccf9ffc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:24.332117 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff942b63-3fc0-4784-97af-bc955ccf9ffc service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:24.341204 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1251/2507] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:21:24 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:21:33.292773 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef6b05-e30c-46ef-a312-807d7a0a802a service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:33.354460 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef6b05-e30c-46ef-a312-807d7a0a802a service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:33.358958 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e4ba8-9bc8-4044-a575-ab7203435d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:33.359606 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e4ba8-9bc8-4044-a575-ab7203435d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:33.359887 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e4ba8-9bc8-4044-a575-ab7203435d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:33.360166 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e4ba8-9bc8-4044-a575-ab7203435d69 admin admin] PATH_INFO: `/v3/users/03986507df5245458411daed053e3ca8` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:33.361274 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70e4ba8-9bc8-4044-a575-ab7203435d69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03986507df5245458411daed053e3ca8)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:33.363229 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70e4ba8-9bc8-4044-a575-ab7203435d69 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:33.529088 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f70e4ba8-9bc8-4044-a575-ab7203435d69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03986507df5245458411daed053e3ca8'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:33.535666 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f70e4ba8-9bc8-4044-a575-ab7203435d69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03986507df5245458411daed053e3ca8'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:33.536320 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f70e4ba8-9bc8-4044-a575-ab7203435d69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03986507df5245458411daed053e3ca8'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:33.538706 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:33.544541 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f70e4ba8-9bc8-4044-a575-ab7203435d69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03986507df5245458411daed053e3ca8'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:33.558116 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f70e4ba8-9bc8-4044-a575-ab7203435d69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03986507df5245458411daed053e3ca8'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:33.564069 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1257/2508] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:21:33 2022] DELETE /identity/v3/users/03986507df5245458411daed053e3ca8 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:33.569042 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83fac1-4266-4416-b701-24761d559914 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:33.569561 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83fac1-4266-4416-b701-24761d559914 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:33.569677 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83fac1-4266-4416-b701-24761d559914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:33.589439 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-ea83fac1-4266-4416-b701-24761d559914 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:21:33.590764 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-ea83fac1-4266-4416-b701-24761d559914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:33.639530 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=100 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:33.648633 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1252/2509] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:21:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:21:33.658424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e4ba8-9bc8-4044-a575-ab7203435d69 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:33.659838 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f70e4ba8-9bc8-4044-a575-ab7203435d69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:33.722510 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e4ba8-9bc8-4044-a575-ab7203435d69 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:33.726074 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac2fa5-399b-4ce8-aeb6-ff54500a5534 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:33.726665 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac2fa5-399b-4ce8-aeb6-ff54500a5534 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:33.726937 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac2fa5-399b-4ce8-aeb6-ff54500a5534 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:33.727238 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac2fa5-399b-4ce8-aeb6-ff54500a5534 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:33.736018 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac2fa5-399b-4ce8-aeb6-ff54500a5534 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:33.737455 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac2fa5-399b-4ce8-aeb6-ff54500a5534 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:33.747438 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1258/2510] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:21:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:21:34.013548 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea83fac1-4266-4416-b701-24761d559914 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:34.015240 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-ea83fac1-4266-4416-b701-24761d559914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:34.029505 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea83fac1-4266-4416-b701-24761d559914 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:34.034046 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628cbd45-4cf1-4a86-8f89-b67e27086dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:34.034830 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628cbd45-4cf1-4a86-8f89-b67e27086dcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:34.035262 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628cbd45-4cf1-4a86-8f89-b67e27086dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:34.035537 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628cbd45-4cf1-4a86-8f89-b67e27086dcd admin admin] PATH_INFO: `/v3/projects/effd0bd0a8894e29918fd3776d71ccad` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:34.037185 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628cbd45-4cf1-4a86-8f89-b67e27086dcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=effd0bd0a8894e29918fd3776d71ccad)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:34.039968 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628cbd45-4cf1-4a86-8f89-b67e27086dcd admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:34.086350 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-628cbd45-4cf1-4a86-8f89-b67e27086dcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'effd0bd0a8894e29918fd3776d71ccad', 'request_id': 'req-628cbd45-4cf1-4a86-8f89-b67e27086dcd'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:34.091750 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-628cbd45-4cf1-4a86-8f89-b67e27086dcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'effd0bd0a8894e29918fd3776d71ccad', 'request_id': 'req-628cbd45-4cf1-4a86-8f89-b67e27086dcd'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:34.097680 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-628cbd45-4cf1-4a86-8f89-b67e27086dcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'effd0bd0a8894e29918fd3776d71ccad', 'request_id': 'req-628cbd45-4cf1-4a86-8f89-b67e27086dcd'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:34.109960 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-628cbd45-4cf1-4a86-8f89-b67e27086dcd admin admin] The token cache is being invalidate because project effd0bd0a8894e29918fd3776d71ccad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:21:34.110250 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-628cbd45-4cf1-4a86-8f89-b67e27086dcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:34.110886 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-628cbd45-4cf1-4a86-8f89-b67e27086dcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:34.112377 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1253/2511] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:21:34 2022] DELETE /identity/v3/projects/effd0bd0a8894e29918fd3776d71ccad => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:34.116382 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac2fa5-399b-4ce8-aeb6-ff54500a5534 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:34.125723 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac2fa5-399b-4ce8-aeb6-ff54500a5534 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:34.179091 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa6ed8-f4df-425c-8b9b-e1c761e57f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:34.179705 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa6ed8-f4df-425c-8b9b-e1c761e57f1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:34.179977 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa6ed8-f4df-425c-8b9b-e1c761e57f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:34.180241 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa6ed8-f4df-425c-8b9b-e1c761e57f1f admin admin] PATH_INFO: `/v3/users/91dbd4e9f7844166b3d4fb42c9442f37` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:34.181318 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa6ed8-f4df-425c-8b9b-e1c761e57f1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=91dbd4e9f7844166b3d4fb42c9442f37)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:34.183247 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa6ed8-f4df-425c-8b9b-e1c761e57f1f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:34.231779 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-8efa6ed8-f4df-425c-8b9b-e1c761e57f1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91dbd4e9f7844166b3d4fb42c9442f37'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:34.238331 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-8efa6ed8-f4df-425c-8b9b-e1c761e57f1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91dbd4e9f7844166b3d4fb42c9442f37'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:34.238993 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-8efa6ed8-f4df-425c-8b9b-e1c761e57f1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91dbd4e9f7844166b3d4fb42c9442f37'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:34.246776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-8efa6ed8-f4df-425c-8b9b-e1c761e57f1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91dbd4e9f7844166b3d4fb42c9442f37'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:34.258756 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-8efa6ed8-f4df-425c-8b9b-e1c761e57f1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91dbd4e9f7844166b3d4fb42c9442f37'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:34.260128 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:34.265455 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1259/2512] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:21:34 2022] DELETE /identity/v3/users/91dbd4e9f7844166b3d4fb42c9442f37 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:34.525473 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628cbd45-4cf1-4a86-8f89-b67e27086dcd admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:34.526951 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-628cbd45-4cf1-4a86-8f89-b67e27086dcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:34.541727 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628cbd45-4cf1-4a86-8f89-b67e27086dcd admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:34.598008 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683d03d4-3f57-4ebe-95e6-c4487b1c6236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:34.598679 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d03d4-3f57-4ebe-95e6-c4487b1c6236 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:34.598894 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d03d4-3f57-4ebe-95e6-c4487b1c6236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:34.599195 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d03d4-3f57-4ebe-95e6-c4487b1c6236 admin admin] PATH_INFO: `/v3/projects/b899f578308b4498aaab98a78db19875` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:34.600293 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683d03d4-3f57-4ebe-95e6-c4487b1c6236 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b899f578308b4498aaab98a78db19875)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:34.602177 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683d03d4-3f57-4ebe-95e6-c4487b1c6236 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:34.646273 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-683d03d4-3f57-4ebe-95e6-c4487b1c6236 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b899f578308b4498aaab98a78db19875', 'request_id': 'req-683d03d4-3f57-4ebe-95e6-c4487b1c6236'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:34.651605 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-683d03d4-3f57-4ebe-95e6-c4487b1c6236 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b899f578308b4498aaab98a78db19875', 'request_id': 'req-683d03d4-3f57-4ebe-95e6-c4487b1c6236'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:34.657371 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-683d03d4-3f57-4ebe-95e6-c4487b1c6236 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b899f578308b4498aaab98a78db19875', 'request_id': 'req-683d03d4-3f57-4ebe-95e6-c4487b1c6236'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:34.669359 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-683d03d4-3f57-4ebe-95e6-c4487b1c6236 admin admin] The token cache is being invalidate because project b899f578308b4498aaab98a78db19875 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:21:34.669648 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-683d03d4-3f57-4ebe-95e6-c4487b1c6236 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:34.670269 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-683d03d4-3f57-4ebe-95e6-c4487b1c6236 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:34.671810 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1254/2513] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:21:34 2022] DELETE /identity/v3/projects/b899f578308b4498aaab98a78db19875 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:34.678156 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a82110-0e87-49e5-bafa-733704468c4d None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:34.678647 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a82110-0e87-49e5-bafa-733704468c4d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:34.678766 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a82110-0e87-49e5-bafa-733704468c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:34.698071 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-a7a82110-0e87-49e5-bafa-733704468c4d None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:21:34.699330 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a7a82110-0e87-49e5-bafa-733704468c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:34.755748 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1260/2514] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:21:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:21:34.763528 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f6235-bf8b-4440-b3ee-319b30fb7c74 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:34.763817 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f6235-bf8b-4440-b3ee-319b30fb7c74 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:34.764087 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f6235-bf8b-4440-b3ee-319b30fb7c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:34.783195 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-102f6235-bf8b-4440-b3ee-319b30fb7c74 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:21:34.784399 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-102f6235-bf8b-4440-b3ee-319b30fb7c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:34.792141 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1255/2515] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:21:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:21:34.796273 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a82110-0e87-49e5-bafa-733704468c4d None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:34.806392 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a82110-0e87-49e5-bafa-733704468c4d None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:34.810892 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81170d3a-9c17-4518-afb3-70aae3bbfa1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:34.811522 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81170d3a-9c17-4518-afb3-70aae3bbfa1b admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:34.811801 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81170d3a-9c17-4518-afb3-70aae3bbfa1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:34.812069 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81170d3a-9c17-4518-afb3-70aae3bbfa1b admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:34.812707 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81170d3a-9c17-4518-afb3-70aae3bbfa1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:21:34.813003 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81170d3a-9c17-4518-afb3-70aae3bbfa1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:34.814586 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81170d3a-9c17-4518-afb3-70aae3bbfa1b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:34.826346 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1261/2516] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:21:34.830699 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f6235-bf8b-4440-b3ee-319b30fb7c74 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:34.839804 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f6235-bf8b-4440-b3ee-319b30fb7c74 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:34.844388 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e264c-e090-439b-8388-cf7a8b9a36b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:34.844971 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e264c-e090-439b-8388-cf7a8b9a36b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:34.845243 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e264c-e090-439b-8388-cf7a8b9a36b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:34.845508 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e264c-e090-439b-8388-cf7a8b9a36b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:34.846133 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e264c-e090-439b-8388-cf7a8b9a36b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:34.848126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e264c-e090-439b-8388-cf7a8b9a36b6 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:34.852972 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:34.876835 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1256/2517] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:21:34 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:21:34.880658 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81170d3a-9c17-4518-afb3-70aae3bbfa1b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:34.890407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81170d3a-9c17-4518-afb3-70aae3bbfa1b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:34.951936 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec9fd4-5959-4d5c-93b0-85c92b102e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:34.952575 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec9fd4-5959-4d5c-93b0-85c92b102e1c admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:34.952851 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec9fd4-5959-4d5c-93b0-85c92b102e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:34.953117 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec9fd4-5959-4d5c-93b0-85c92b102e1c admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:34.953732 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ec9fd4-5959-4d5c-93b0-85c92b102e1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:34.955743 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ec9fd4-5959-4d5c-93b0-85c92b102e1c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:34.977050 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1262/2518] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:21:34 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:21:34.981061 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e264c-e090-439b-8388-cf7a8b9a36b6 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:34.992088 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e264c-e090-439b-8388-cf7a8b9a36b6 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:34.997032 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118eed8f-93e9-4103-b2cb-c1ac41ce242e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:34.997634 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118eed8f-93e9-4103-b2cb-c1ac41ce242e admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:34.997915 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118eed8f-93e9-4103-b2cb-c1ac41ce242e admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:34.998214 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118eed8f-93e9-4103-b2cb-c1ac41ce242e admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:34.998884 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118eed8f-93e9-4103-b2cb-c1ac41ce242e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:35.000534 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118eed8f-93e9-4103-b2cb-c1ac41ce242e admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:35.011628 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1257/2519] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:21:35.015633 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec9fd4-5959-4d5c-93b0-85c92b102e1c admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:35.025318 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec9fd4-5959-4d5c-93b0-85c92b102e1c admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:35.029844 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a0818-8525-476b-9a0e-e9dd9b69b20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:35.030426 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a0818-8525-476b-9a0e-e9dd9b69b20a admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:35.030699 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a0818-8525-476b-9a0e-e9dd9b69b20a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:35.030965 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a0818-8525-476b-9a0e-e9dd9b69b20a admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:35.031625 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a0818-8525-476b-9a0e-e9dd9b69b20a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:35.033212 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a0818-8525-476b-9a0e-e9dd9b69b20a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:35.043290 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1263/2520] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:21:35.047273 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118eed8f-93e9-4103-b2cb-c1ac41ce242e admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:35.058971 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118eed8f-93e9-4103-b2cb-c1ac41ce242e admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:35.063885 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591356ad-23a2-497a-90ee-cd2051b627b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:35.064535 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591356ad-23a2-497a-90ee-cd2051b627b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:35.064817 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591356ad-23a2-497a-90ee-cd2051b627b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:35.065088 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591356ad-23a2-497a-90ee-cd2051b627b1 admin admin] PATH_INFO: `/v3/projects/fb70f646a10949d3afd3b981f9bb004c/users/9676292ae961474aa2952f88ae1ec220/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:35.079496 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591356ad-23a2-497a-90ee-cd2051b627b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb70f646a10949d3afd3b981f9bb004c, user_id=9676292ae961474aa2952f88ae1ec220, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:35.082126 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591356ad-23a2-497a-90ee-cd2051b627b1 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:35.094873 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1258/2521] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:21:35 2022] PUT /identity/v3/projects/fb70f646a10949d3afd3b981f9bb004c/users/9676292ae961474aa2952f88ae1ec220/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:35.158184 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ee1e9-67b0-4b68-886f-a3b600edf275 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:35.158452 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ee1e9-67b0-4b68-886f-a3b600edf275 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:35.158748 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20ee1e9-67b0-4b68-886f-a3b600edf275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:35.186338 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f20ee1e9-67b0-4b68-886f-a3b600edf275 None None] MFA Rules not processed for user `9676292ae961474aa2952f88ae1ec220`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:21:35.187497 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f20ee1e9-67b0-4b68-886f-a3b600edf275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:35.238804 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:35.250024 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1264/2522] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:21:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:21:35.254430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591356ad-23a2-497a-90ee-cd2051b627b1 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:35.313493 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591356ad-23a2-497a-90ee-cd2051b627b1 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:35.318589 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237cddf7-d817-43d3-9bfb-eb3256bf23ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:35.319205 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237cddf7-d817-43d3-9bfb-eb3256bf23ab admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:35.319483 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237cddf7-d817-43d3-9bfb-eb3256bf23ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:35.319749 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237cddf7-d817-43d3-9bfb-eb3256bf23ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:35.320361 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237cddf7-d817-43d3-9bfb-eb3256bf23ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:35.322249 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237cddf7-d817-43d3-9bfb-eb3256bf23ab admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:35.347485 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1259/2523] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:21:35 2022] POST /identity/v3/projects => generated 383 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:21:35.351558 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ee1e9-67b0-4b68-886f-a3b600edf275 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:35.361867 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20ee1e9-67b0-4b68-886f-a3b600edf275 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:35.416599 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7257a83-e9d6-42ea-9ec8-2061483ccc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:35.417192 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7257a83-e9d6-42ea-9ec8-2061483ccc6d admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:35.417473 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7257a83-e9d6-42ea-9ec8-2061483ccc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:35.417821 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7257a83-e9d6-42ea-9ec8-2061483ccc6d admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:35.418550 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7257a83-e9d6-42ea-9ec8-2061483ccc6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:35.420583 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7257a83-e9d6-42ea-9ec8-2061483ccc6d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:35.441410 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1265/2524] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:21:35 2022] POST /identity/v3/users => generated 298 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:21:35.445800 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237cddf7-d817-43d3-9bfb-eb3256bf23ab admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:35.455884 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237cddf7-d817-43d3-9bfb-eb3256bf23ab admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:35.460473 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11cf6a6-84e9-45e5-ae24-b85b3bfbb2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:35.461067 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11cf6a6-84e9-45e5-ae24-b85b3bfbb2c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:35.461349 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11cf6a6-84e9-45e5-ae24-b85b3bfbb2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:35.461623 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11cf6a6-84e9-45e5-ae24-b85b3bfbb2c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:35.462259 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11cf6a6-84e9-45e5-ae24-b85b3bfbb2c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:35.463890 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11cf6a6-84e9-45e5-ae24-b85b3bfbb2c3 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:35.474135 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1260/2525] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:21:35.477958 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7257a83-e9d6-42ea-9ec8-2061483ccc6d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:35.487751 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7257a83-e9d6-42ea-9ec8-2061483ccc6d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:35.492884 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b372578-5f00-4f74-8a10-52af8874a56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:35.493516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b372578-5f00-4f74-8a10-52af8874a56f admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:35.493798 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b372578-5f00-4f74-8a10-52af8874a56f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:35.494076 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b372578-5f00-4f74-8a10-52af8874a56f admin admin] PATH_INFO: `/v3/projects/252774e4ed7f4197b84e3a2c6e9efc72/users/8c1dfe8fcd794238b77bedec07baf7d1/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:35.509652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b372578-5f00-4f74-8a10-52af8874a56f admin admin] RBAC: Authorizing `identity:create_grant(project_id=252774e4ed7f4197b84e3a2c6e9efc72, user_id=8c1dfe8fcd794238b77bedec07baf7d1, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:35.512309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b372578-5f00-4f74-8a10-52af8874a56f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:35.524328 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1266/2526] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:21:35 2022] PUT /identity/v3/projects/252774e4ed7f4197b84e3a2c6e9efc72/users/8c1dfe8fcd794238b77bedec07baf7d1/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:35.528213 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11cf6a6-84e9-45e5-ae24-b85b3bfbb2c3 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:35.538135 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11cf6a6-84e9-45e5-ae24-b85b3bfbb2c3 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:35.593255 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919378e-c387-40cc-ba97-5b985a20c0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:35.593854 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919378e-c387-40cc-ba97-5b985a20c0f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:35.594136 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919378e-c387-40cc-ba97-5b985a20c0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:35.594412 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919378e-c387-40cc-ba97-5b985a20c0f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:35.595049 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9919378e-c387-40cc-ba97-5b985a20c0f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:35.596696 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9919378e-c387-40cc-ba97-5b985a20c0f6 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:35.606625 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1261/2527] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:21:35.610620 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b372578-5f00-4f74-8a10-52af8874a56f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:35.621217 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b372578-5f00-4f74-8a10-52af8874a56f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:35.625706 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea4f0e3-73a1-41ac-b8bb-51fb50f53690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:35.626335 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea4f0e3-73a1-41ac-b8bb-51fb50f53690 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:35.626611 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea4f0e3-73a1-41ac-b8bb-51fb50f53690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:35.626878 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea4f0e3-73a1-41ac-b8bb-51fb50f53690 admin admin] PATH_INFO: `/v3/projects/252774e4ed7f4197b84e3a2c6e9efc72/users/8c1dfe8fcd794238b77bedec07baf7d1/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:35.628782 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea4f0e3-73a1-41ac-b8bb-51fb50f53690 admin admin] RBAC: Authorizing `identity:create_grant(project_id=252774e4ed7f4197b84e3a2c6e9efc72, user_id=8c1dfe8fcd794238b77bedec07baf7d1, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:35.631423 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea4f0e3-73a1-41ac-b8bb-51fb50f53690 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:35.641538 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1267/2528] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:21:35 2022] PUT /identity/v3/projects/252774e4ed7f4197b84e3a2c6e9efc72/users/8c1dfe8fcd794238b77bedec07baf7d1/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:35.651384 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a23e80-e310-42ff-97e0-40ca2327da02 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:35.651675 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a23e80-e310-42ff-97e0-40ca2327da02 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:35.651951 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a23e80-e310-42ff-97e0-40ca2327da02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:35.680213 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-68a23e80-e310-42ff-97e0-40ca2327da02 None None] MFA Rules not processed for user `8c1dfe8fcd794238b77bedec07baf7d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:21:35.681429 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-68a23e80-e310-42ff-97e0-40ca2327da02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:35.689286 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:35.749758 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1262/2529] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:21:35 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:21:35.760387 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea4f0e3-73a1-41ac-b8bb-51fb50f53690 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:35.762080 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4ea4f0e3-73a1-41ac-b8bb-51fb50f53690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:35.817850 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea4f0e3-73a1-41ac-b8bb-51fb50f53690 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:35.821555 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4135ee3-377a-474d-976b-26d9684b9a38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:35.822149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4135ee3-377a-474d-976b-26d9684b9a38 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:35.822424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4135ee3-377a-474d-976b-26d9684b9a38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:35.822691 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4135ee3-377a-474d-976b-26d9684b9a38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:35.831559 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4135ee3-377a-474d-976b-26d9684b9a38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:35.832965 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4135ee3-377a-474d-976b-26d9684b9a38 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:35.842075 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1268/2530] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:21:35 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:21:35.946203 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a23e80-e310-42ff-97e0-40ca2327da02 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:35.955853 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a23e80-e310-42ff-97e0-40ca2327da02 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:35.959963 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b7bf8-dfa7-4de9-a894-b05dab3caa46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:35.960554 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b7bf8-dfa7-4de9-a894-b05dab3caa46 service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:35.960832 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b7bf8-dfa7-4de9-a894-b05dab3caa46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:35.961104 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b7bf8-dfa7-4de9-a894-b05dab3caa46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:36.015125 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b7bf8-dfa7-4de9-a894-b05dab3caa46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:36.016497 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b7bf8-dfa7-4de9-a894-b05dab3caa46 service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:36.025827 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1263/2531] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:21:35 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:21:38.597597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4135ee3-377a-474d-976b-26d9684b9a38 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:38.643712 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:38.659024 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4135ee3-377a-474d-976b-26d9684b9a38 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:38.663690 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5059340-baa4-4376-a54a-e33c3f3f39e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:38.664309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5059340-baa4-4376-a54a-e33c3f3f39e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:38.664589 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5059340-baa4-4376-a54a-e33c3f3f39e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:38.664857 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5059340-baa4-4376-a54a-e33c3f3f39e9 admin admin] PATH_INFO: `/v3/users/9676292ae961474aa2952f88ae1ec220` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:38.665978 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5059340-baa4-4376-a54a-e33c3f3f39e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9676292ae961474aa2952f88ae1ec220)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:38.667968 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5059340-baa4-4376-a54a-e33c3f3f39e9 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:38.713309 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d5059340-baa4-4376-a54a-e33c3f3f39e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9676292ae961474aa2952f88ae1ec220'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:38.720052 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d5059340-baa4-4376-a54a-e33c3f3f39e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9676292ae961474aa2952f88ae1ec220'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:38.720721 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d5059340-baa4-4376-a54a-e33c3f3f39e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9676292ae961474aa2952f88ae1ec220'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:38.727798 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d5059340-baa4-4376-a54a-e33c3f3f39e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9676292ae961474aa2952f88ae1ec220'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:38.739678 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d5059340-baa4-4376-a54a-e33c3f3f39e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9676292ae961474aa2952f88ae1ec220'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:38.745491 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1269/2532] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:21:38 2022] DELETE /identity/v3/users/9676292ae961474aa2952f88ae1ec220 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:38.749968 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c372e4-d9c8-45a6-9e42-76dc3c6d0e1f None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:38.750437 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c372e4-d9c8-45a6-9e42-76dc3c6d0e1f None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:38.750574 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c372e4-d9c8-45a6-9e42-76dc3c6d0e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:38.769917 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-f6c372e4-d9c8-45a6-9e42-76dc3c6d0e1f None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:21:38.771720 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-f6c372e4-d9c8-45a6-9e42-76dc3c6d0e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:38.835180 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1264/2533] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:21:38.845108 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5059340-baa4-4376-a54a-e33c3f3f39e9 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:38.846852 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d5059340-baa4-4376-a54a-e33c3f3f39e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:38.902441 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5059340-baa4-4376-a54a-e33c3f3f39e9 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:38.906019 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd56c4c7-5801-4517-8da3-8eee72c01cd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:38.906615 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56c4c7-5801-4517-8da3-8eee72c01cd1 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:38.906888 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56c4c7-5801-4517-8da3-8eee72c01cd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:38.907178 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56c4c7-5801-4517-8da3-8eee72c01cd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:38.916107 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd56c4c7-5801-4517-8da3-8eee72c01cd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:38.917543 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd56c4c7-5801-4517-8da3-8eee72c01cd1 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:38.927820 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1270/2534] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:21:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:21:39.188570 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c372e4-d9c8-45a6-9e42-76dc3c6d0e1f None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:39.189997 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-f6c372e4-d9c8-45a6-9e42-76dc3c6d0e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:39.203171 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c372e4-d9c8-45a6-9e42-76dc3c6d0e1f None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:39.207829 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc625652-319e-412d-803b-791d1ae4ddc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:39.208464 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc625652-319e-412d-803b-791d1ae4ddc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:39.208744 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc625652-319e-412d-803b-791d1ae4ddc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:39.209017 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc625652-319e-412d-803b-791d1ae4ddc3 admin admin] PATH_INFO: `/v3/projects/fb70f646a10949d3afd3b981f9bb004c` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:39.210200 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc625652-319e-412d-803b-791d1ae4ddc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb70f646a10949d3afd3b981f9bb004c)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:39.212176 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc625652-319e-412d-803b-791d1ae4ddc3 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:39.251520 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:39.259289 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-bc625652-319e-412d-803b-791d1ae4ddc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb70f646a10949d3afd3b981f9bb004c', 'request_id': 'req-bc625652-319e-412d-803b-791d1ae4ddc3'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:39.264729 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-bc625652-319e-412d-803b-791d1ae4ddc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb70f646a10949d3afd3b981f9bb004c', 'request_id': 'req-bc625652-319e-412d-803b-791d1ae4ddc3'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:39.270711 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-bc625652-319e-412d-803b-791d1ae4ddc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb70f646a10949d3afd3b981f9bb004c', 'request_id': 'req-bc625652-319e-412d-803b-791d1ae4ddc3'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:39.282714 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-bc625652-319e-412d-803b-791d1ae4ddc3 admin admin] The token cache is being invalidate because project fb70f646a10949d3afd3b981f9bb004c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:21:39.283004 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-bc625652-319e-412d-803b-791d1ae4ddc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:39.283768 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-bc625652-319e-412d-803b-791d1ae4ddc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:39.285221 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1265/2535] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:21:39 2022] DELETE /identity/v3/projects/fb70f646a10949d3afd3b981f9bb004c => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:39.289982 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd56c4c7-5801-4517-8da3-8eee72c01cd1 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:39.299516 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd56c4c7-5801-4517-8da3-8eee72c01cd1 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:39.353607 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de010355-0c18-463f-9bf1-cd9e8b3a0ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:39.354149 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de010355-0c18-463f-9bf1-cd9e8b3a0ab2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:39.354426 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de010355-0c18-463f-9bf1-cd9e8b3a0ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:39.354698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de010355-0c18-463f-9bf1-cd9e8b3a0ab2 admin admin] PATH_INFO: `/v3/users/8c1dfe8fcd794238b77bedec07baf7d1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:39.355830 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de010355-0c18-463f-9bf1-cd9e8b3a0ab2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c1dfe8fcd794238b77bedec07baf7d1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:39.357720 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de010355-0c18-463f-9bf1-cd9e8b3a0ab2 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:39.370201 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:39.403134 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-de010355-0c18-463f-9bf1-cd9e8b3a0ab2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c1dfe8fcd794238b77bedec07baf7d1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:39.409858 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-de010355-0c18-463f-9bf1-cd9e8b3a0ab2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c1dfe8fcd794238b77bedec07baf7d1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:39.410571 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-de010355-0c18-463f-9bf1-cd9e8b3a0ab2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c1dfe8fcd794238b77bedec07baf7d1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:39.418275 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-de010355-0c18-463f-9bf1-cd9e8b3a0ab2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c1dfe8fcd794238b77bedec07baf7d1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:39.430855 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-de010355-0c18-463f-9bf1-cd9e8b3a0ab2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c1dfe8fcd794238b77bedec07baf7d1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:39.436709 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1271/2536] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:21:39 2022] DELETE /identity/v3/users/8c1dfe8fcd794238b77bedec07baf7d1 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:39.698111 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc625652-319e-412d-803b-791d1ae4ddc3 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:39.699584 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-bc625652-319e-412d-803b-791d1ae4ddc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:39.714312 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc625652-319e-412d-803b-791d1ae4ddc3 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:39.770620 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75a9f4-9a2b-48d6-9ae6-dc94efea8929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:39.771777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75a9f4-9a2b-48d6-9ae6-dc94efea8929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:39.772060 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75a9f4-9a2b-48d6-9ae6-dc94efea8929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:39.772332 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75a9f4-9a2b-48d6-9ae6-dc94efea8929 admin admin] PATH_INFO: `/v3/projects/252774e4ed7f4197b84e3a2c6e9efc72` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:39.773436 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca75a9f4-9a2b-48d6-9ae6-dc94efea8929 admin admin] RBAC: Authorizing `identity:delete_project(project_id=252774e4ed7f4197b84e3a2c6e9efc72)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:39.775371 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca75a9f4-9a2b-48d6-9ae6-dc94efea8929 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:39.821858 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-ca75a9f4-9a2b-48d6-9ae6-dc94efea8929 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '252774e4ed7f4197b84e3a2c6e9efc72', 'request_id': 'req-ca75a9f4-9a2b-48d6-9ae6-dc94efea8929'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:39.827514 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-ca75a9f4-9a2b-48d6-9ae6-dc94efea8929 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '252774e4ed7f4197b84e3a2c6e9efc72', 'request_id': 'req-ca75a9f4-9a2b-48d6-9ae6-dc94efea8929'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:39.834274 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-ca75a9f4-9a2b-48d6-9ae6-dc94efea8929 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '252774e4ed7f4197b84e3a2c6e9efc72', 'request_id': 'req-ca75a9f4-9a2b-48d6-9ae6-dc94efea8929'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:39.847150 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-ca75a9f4-9a2b-48d6-9ae6-dc94efea8929 admin admin] The token cache is being invalidate because project 252774e4ed7f4197b84e3a2c6e9efc72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:21:39.847459 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-ca75a9f4-9a2b-48d6-9ae6-dc94efea8929 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:39.848078 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-ca75a9f4-9a2b-48d6-9ae6-dc94efea8929 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:39.850170 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1266/2537] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:21:39 2022] DELETE /identity/v3/projects/252774e4ed7f4197b84e3a2c6e9efc72 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:39.855037 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f6774-4482-4525-860d-384561435734 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:39.855514 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f6774-4482-4525-860d-384561435734 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:39.855668 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f6774-4482-4525-860d-384561435734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:39.875573 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-283f6774-4482-4525-860d-384561435734 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:21:39.876698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-283f6774-4482-4525-860d-384561435734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:39.934811 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1272/2538] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 13 13:21:39.942527 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f6ad0-4534-49c7-a1c6-aa4958da7a90 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:39.943499 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f6ad0-4534-49c7-a1c6-aa4958da7a90 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:39.943773 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9f6ad0-4534-49c7-a1c6-aa4958da7a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:39.962674 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-0d9f6ad0-4534-49c7-a1c6-aa4958da7a90 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:21:39.963890 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-0d9f6ad0-4534-49c7-a1c6-aa4958da7a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:39.971541 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1267/2539] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:21:39.975512 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f6774-4482-4525-860d-384561435734 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:39.984945 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f6774-4482-4525-860d-384561435734 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:39.989424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e1275e-f606-4388-bd41-2a2ef345be27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:39.990524 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1275e-f606-4388-bd41-2a2ef345be27 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:39.990799 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1275e-f606-4388-bd41-2a2ef345be27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:39.991081 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1275e-f606-4388-bd41-2a2ef345be27 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:39.991730 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e1275e-f606-4388-bd41-2a2ef345be27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:21:39.992028 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e1275e-f606-4388-bd41-2a2ef345be27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:39.993587 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e1275e-f606-4388-bd41-2a2ef345be27 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:40.004694 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1273/2540] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:21:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:21:40.009365 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f6ad0-4534-49c7-a1c6-aa4958da7a90 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:40.018454 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9f6ad0-4534-49c7-a1c6-aa4958da7a90 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:40.022999 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aac5b0-8419-4075-9ae5-f8dde32f6774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:40.023613 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aac5b0-8419-4075-9ae5-f8dde32f6774 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:40.023892 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aac5b0-8419-4075-9ae5-f8dde32f6774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:40.024159 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aac5b0-8419-4075-9ae5-f8dde32f6774 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:40.024793 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aac5b0-8419-4075-9ae5-f8dde32f6774 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:40.026700 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aac5b0-8419-4075-9ae5-f8dde32f6774 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:40.053797 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1268/2541] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:21:40 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 13 13:21:40.057983 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e1275e-f606-4388-bd41-2a2ef345be27 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:40.067676 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e1275e-f606-4388-bd41-2a2ef345be27 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:40.121814 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b5413-6c75-428c-8989-493725275520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:40.122467 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b5413-6c75-428c-8989-493725275520 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:40.122883 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b5413-6c75-428c-8989-493725275520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:40.123262 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b5413-6c75-428c-8989-493725275520 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:40.123900 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b5413-6c75-428c-8989-493725275520 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:40.125850 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b5413-6c75-428c-8989-493725275520 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:40.146052 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1274/2542] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:21:40 2022] POST /identity/v3/users => generated 292 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:21:40.150245 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aac5b0-8419-4075-9ae5-f8dde32f6774 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:40.160314 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aac5b0-8419-4075-9ae5-f8dde32f6774 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:40.165344 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e34b8-3d5c-4232-a6ad-71594c217d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:40.165937 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e34b8-3d5c-4232-a6ad-71594c217d19 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:40.166205 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e34b8-3d5c-4232-a6ad-71594c217d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:40.166474 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e34b8-3d5c-4232-a6ad-71594c217d19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:40.167136 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e34b8-3d5c-4232-a6ad-71594c217d19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:40.168713 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e34b8-3d5c-4232-a6ad-71594c217d19 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:40.179296 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1269/2543] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:21:40.183027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b5413-6c75-428c-8989-493725275520 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:40.192688 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b5413-6c75-428c-8989-493725275520 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:40.197229 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d509d8-2c02-4e74-aa0d-c6b64479f585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:40.197810 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d509d8-2c02-4e74-aa0d-c6b64479f585 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:40.198088 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d509d8-2c02-4e74-aa0d-c6b64479f585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:40.198357 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d509d8-2c02-4e74-aa0d-c6b64479f585 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:40.198988 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d509d8-2c02-4e74-aa0d-c6b64479f585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:40.200573 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d509d8-2c02-4e74-aa0d-c6b64479f585 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:40.201751 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:40.211981 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1275/2544] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:21:40.216124 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e34b8-3d5c-4232-a6ad-71594c217d19 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:40.226387 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e34b8-3d5c-4232-a6ad-71594c217d19 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:40.231192 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc53e33-0d4a-4656-b107-2d49cf4b7717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:40.231848 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc53e33-0d4a-4656-b107-2d49cf4b7717 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:40.232123 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc53e33-0d4a-4656-b107-2d49cf4b7717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:40.232393 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc53e33-0d4a-4656-b107-2d49cf4b7717 admin admin] PATH_INFO: `/v3/projects/618c9b539fbe4c3dbc1ecfb83e47118a/users/44ecb8ca736a4e8b9dfb172e7e01deec/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:40.247945 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc53e33-0d4a-4656-b107-2d49cf4b7717 admin admin] RBAC: Authorizing `identity:create_grant(project_id=618c9b539fbe4c3dbc1ecfb83e47118a, user_id=44ecb8ca736a4e8b9dfb172e7e01deec, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:40.250563 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc53e33-0d4a-4656-b107-2d49cf4b7717 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:40.252732 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:40.264921 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1270/2545] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:21:40 2022] PUT /identity/v3/projects/618c9b539fbe4c3dbc1ecfb83e47118a/users/44ecb8ca736a4e8b9dfb172e7e01deec/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:40.276266 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5418d-e5a2-4844-b832-d594ec112ca6 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:40.276555 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5418d-e5a2-4844-b832-d594ec112ca6 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:40.276832 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5418d-e5a2-4844-b832-d594ec112ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:40.305686 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d1b5418d-e5a2-4844-b832-d594ec112ca6 None None] MFA Rules not processed for user `44ecb8ca736a4e8b9dfb172e7e01deec`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:21:40.306888 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d1b5418d-e5a2-4844-b832-d594ec112ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:40.368034 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1276/2546] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:21:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:21:40.372489 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc53e33-0d4a-4656-b107-2d49cf4b7717 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:40.432119 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc53e33-0d4a-4656-b107-2d49cf4b7717 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:40.436629 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c1b25-011c-4d4a-8082-eda39257e387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:40.437216 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c1b25-011c-4d4a-8082-eda39257e387 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:40.437495 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c1b25-011c-4d4a-8082-eda39257e387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:40.437764 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c1b25-011c-4d4a-8082-eda39257e387 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:40.438395 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c1b25-011c-4d4a-8082-eda39257e387 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:40.440337 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c1b25-011c-4d4a-8082-eda39257e387 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:40.465466 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1271/2547] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:21:40 2022] POST /identity/v3/projects => generated 381 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:21:40.469412 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b5418d-e5a2-4844-b832-d594ec112ca6 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:40.480366 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b5418d-e5a2-4844-b832-d594ec112ca6 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:40.534067 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd8550-ff3f-41f2-8d05-7461c9626df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:40.534664 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd8550-ff3f-41f2-8d05-7461c9626df7 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:40.534941 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd8550-ff3f-41f2-8d05-7461c9626df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:40.535243 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd8550-ff3f-41f2-8d05-7461c9626df7 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:40.535862 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd8550-ff3f-41f2-8d05-7461c9626df7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:40.538329 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd8550-ff3f-41f2-8d05-7461c9626df7 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:40.558820 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1277/2548] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:21:40 2022] POST /identity/v3/users => generated 297 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:21:40.562817 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c1b25-011c-4d4a-8082-eda39257e387 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:40.573070 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c1b25-011c-4d4a-8082-eda39257e387 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:40.577716 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce684bd-206c-4240-8376-9c1384e058d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:40.578309 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce684bd-206c-4240-8376-9c1384e058d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:40.578591 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce684bd-206c-4240-8376-9c1384e058d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:40.578864 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce684bd-206c-4240-8376-9c1384e058d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:40.579541 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce684bd-206c-4240-8376-9c1384e058d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:40.581131 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce684bd-206c-4240-8376-9c1384e058d7 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:40.591932 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1272/2549] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:21:40.595929 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd8550-ff3f-41f2-8d05-7461c9626df7 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:40.606092 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd8550-ff3f-41f2-8d05-7461c9626df7 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:40.610580 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbbf66e-42c8-4c67-825f-bbc3606da723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:40.611247 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbf66e-42c8-4c67-825f-bbc3606da723 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:40.611526 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbf66e-42c8-4c67-825f-bbc3606da723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:40.611795 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbf66e-42c8-4c67-825f-bbc3606da723 admin admin] PATH_INFO: `/v3/projects/065aa8d6f0e745ca9260f4dcc249f204/users/3ef8e932d5ee49d9884b94f1d6bcf99a/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:40.626476 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbbf66e-42c8-4c67-825f-bbc3606da723 admin admin] RBAC: Authorizing `identity:create_grant(project_id=065aa8d6f0e745ca9260f4dcc249f204, user_id=3ef8e932d5ee49d9884b94f1d6bcf99a, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:40.629111 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbbf66e-42c8-4c67-825f-bbc3606da723 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:40.641232 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1278/2550] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:21:40 2022] PUT /identity/v3/projects/065aa8d6f0e745ca9260f4dcc249f204/users/3ef8e932d5ee49d9884b94f1d6bcf99a/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:40.645052 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce684bd-206c-4240-8376-9c1384e058d7 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:40.655349 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce684bd-206c-4240-8376-9c1384e058d7 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:40.711939 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9ae08-1389-4f41-8e22-33fab3fe38cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:40.712463 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9ae08-1389-4f41-8e22-33fab3fe38cd admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:40.712739 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9ae08-1389-4f41-8e22-33fab3fe38cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:40.713010 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9ae08-1389-4f41-8e22-33fab3fe38cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:40.713640 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff9ae08-1389-4f41-8e22-33fab3fe38cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:40.715266 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff9ae08-1389-4f41-8e22-33fab3fe38cd admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:40.724646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1273/2551] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:21:40.728904 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbbf66e-42c8-4c67-825f-bbc3606da723 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:40.738834 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbbf66e-42c8-4c67-825f-bbc3606da723 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:40.744092 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6e65f-7b28-47c4-afb2-923be31420af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:40.744737 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6e65f-7b28-47c4-afb2-923be31420af admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:40.745016 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6e65f-7b28-47c4-afb2-923be31420af admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:40.745288 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6e65f-7b28-47c4-afb2-923be31420af admin admin] PATH_INFO: `/v3/projects/065aa8d6f0e745ca9260f4dcc249f204/users/3ef8e932d5ee49d9884b94f1d6bcf99a/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:40.747301 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b6e65f-7b28-47c4-afb2-923be31420af admin admin] RBAC: Authorizing `identity:create_grant(project_id=065aa8d6f0e745ca9260f4dcc249f204, user_id=3ef8e932d5ee49d9884b94f1d6bcf99a, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:40.749893 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b6e65f-7b28-47c4-afb2-923be31420af admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:40.760507 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1279/2552] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:21:40 2022] PUT /identity/v3/projects/065aa8d6f0e745ca9260f4dcc249f204/users/3ef8e932d5ee49d9884b94f1d6bcf99a/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 13 13:21:40.770756 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fd63e-625e-42e4-ba5a-f335f8602007 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:40.771253 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fd63e-625e-42e4-ba5a-f335f8602007 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:40.771368 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fd63e-625e-42e4-ba5a-f335f8602007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:40.799128 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-9d0fd63e-625e-42e4-ba5a-f335f8602007 None None] MFA Rules not processed for user `3ef8e932d5ee49d9884b94f1d6bcf99a`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:21:40.800354 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-9d0fd63e-625e-42e4-ba5a-f335f8602007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:40.848840 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:40.868746 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1274/2553] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:21:40 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:21:40.879968 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6e65f-7b28-47c4-afb2-923be31420af admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:40.881637 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a5b6e65f-7b28-47c4-afb2-923be31420af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:40.923277 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:40.939905 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6e65f-7b28-47c4-afb2-923be31420af admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:40.943499 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb1345-19f3-4c28-a6ae-131e07c64296 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:40.944090 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb1345-19f3-4c28-a6ae-131e07c64296 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:40.944368 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb1345-19f3-4c28-a6ae-131e07c64296 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:40.944635 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb1345-19f3-4c28-a6ae-131e07c64296 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:40.953222 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfb1345-19f3-4c28-a6ae-131e07c64296 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:40.954630 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfb1345-19f3-4c28-a6ae-131e07c64296 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:40.963726 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1280/2554] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:21:40 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:21:41.035098 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0fd63e-625e-42e4-ba5a-f335f8602007 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:41.045147 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0fd63e-625e-42e4-ba5a-f335f8602007 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:41.048992 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6f7cb8-d303-47ae-b931-f4e8f5f53766 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:41.049580 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f7cb8-d303-47ae-b931-f4e8f5f53766 service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:41.050395 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f7cb8-d303-47ae-b931-f4e8f5f53766 service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:41.050395 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f7cb8-d303-47ae-b931-f4e8f5f53766 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:41.105037 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6f7cb8-d303-47ae-b931-f4e8f5f53766 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:41.106605 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6f7cb8-d303-47ae-b931-f4e8f5f53766 service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:41.115275 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1275/2555] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:21:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 13 13:21:41.374536 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb1345-19f3-4c28-a6ae-131e07c64296 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:41.435386 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb1345-19f3-4c28-a6ae-131e07c64296 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:41.440254 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d266e7-ac8a-442c-9bd7-092a6c99e0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:41.440871 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d266e7-ac8a-442c-9bd7-092a6c99e0d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:41.441147 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d266e7-ac8a-442c-9bd7-092a6c99e0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:41.441417 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d266e7-ac8a-442c-9bd7-092a6c99e0d4 admin admin] PATH_INFO: `/v3/users/44ecb8ca736a4e8b9dfb172e7e01deec` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:41.442510 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d266e7-ac8a-442c-9bd7-092a6c99e0d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44ecb8ca736a4e8b9dfb172e7e01deec)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:41.444467 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d266e7-ac8a-442c-9bd7-092a6c99e0d4 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:41.489698 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-39d266e7-ac8a-442c-9bd7-092a6c99e0d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44ecb8ca736a4e8b9dfb172e7e01deec'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:41.496426 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-39d266e7-ac8a-442c-9bd7-092a6c99e0d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44ecb8ca736a4e8b9dfb172e7e01deec'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:41.497207 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-39d266e7-ac8a-442c-9bd7-092a6c99e0d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44ecb8ca736a4e8b9dfb172e7e01deec'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:41.504484 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-39d266e7-ac8a-442c-9bd7-092a6c99e0d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44ecb8ca736a4e8b9dfb172e7e01deec'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:41.517407 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-39d266e7-ac8a-442c-9bd7-092a6c99e0d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44ecb8ca736a4e8b9dfb172e7e01deec'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:41.523918 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1281/2556] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:21:41 2022] DELETE /identity/v3/users/44ecb8ca736a4e8b9dfb172e7e01deec => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:41.528551 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6e35b-579e-4db2-898f-b0e6b7750772 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:41.528835 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6e35b-579e-4db2-898f-b0e6b7750772 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:41.529103 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6e35b-579e-4db2-898f-b0e6b7750772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:41.548777 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-dbc6e35b-579e-4db2-898f-b0e6b7750772 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:21:41.549966 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-dbc6e35b-579e-4db2-898f-b0e6b7750772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:41.606357 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1276/2557] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:21:41.616025 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d266e7-ac8a-442c-9bd7-092a6c99e0d4 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:41.617381 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-39d266e7-ac8a-442c-9bd7-092a6c99e0d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:41.673414 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d266e7-ac8a-442c-9bd7-092a6c99e0d4 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:41.676953 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc739cc-6cb2-41f2-b269-931e863cc6bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:41.677546 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc739cc-6cb2-41f2-b269-931e863cc6bc service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:41.677823 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc739cc-6cb2-41f2-b269-931e863cc6bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:41.678092 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc739cc-6cb2-41f2-b269-931e863cc6bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:41.686870 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc739cc-6cb2-41f2-b269-931e863cc6bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:41.688303 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc739cc-6cb2-41f2-b269-931e863cc6bc service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:41.698023 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1282/2558] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:21:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:21:41.963802 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6e35b-579e-4db2-898f-b0e6b7750772 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:41.965263 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-dbc6e35b-579e-4db2-898f-b0e6b7750772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:41.978383 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6e35b-579e-4db2-898f-b0e6b7750772 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:41.984604 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c7bad-2541-4d10-9519-7174fe60b998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:41.985235 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c7bad-2541-4d10-9519-7174fe60b998 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:41.985520 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c7bad-2541-4d10-9519-7174fe60b998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:41.985797 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c7bad-2541-4d10-9519-7174fe60b998 admin admin] PATH_INFO: `/v3/projects/618c9b539fbe4c3dbc1ecfb83e47118a` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:41.987504 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c7bad-2541-4d10-9519-7174fe60b998 admin admin] RBAC: Authorizing `identity:delete_project(project_id=618c9b539fbe4c3dbc1ecfb83e47118a)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:41.989807 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c7bad-2541-4d10-9519-7174fe60b998 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:42.037430 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-815c7bad-2541-4d10-9519-7174fe60b998 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '618c9b539fbe4c3dbc1ecfb83e47118a', 'request_id': 'req-815c7bad-2541-4d10-9519-7174fe60b998'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:42.043110 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-815c7bad-2541-4d10-9519-7174fe60b998 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '618c9b539fbe4c3dbc1ecfb83e47118a', 'request_id': 'req-815c7bad-2541-4d10-9519-7174fe60b998'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:42.049406 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-815c7bad-2541-4d10-9519-7174fe60b998 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '618c9b539fbe4c3dbc1ecfb83e47118a', 'request_id': 'req-815c7bad-2541-4d10-9519-7174fe60b998'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:42.061375 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-815c7bad-2541-4d10-9519-7174fe60b998 admin admin] The token cache is being invalidate because project 618c9b539fbe4c3dbc1ecfb83e47118a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:21:42.061651 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-815c7bad-2541-4d10-9519-7174fe60b998 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:42.062266 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-815c7bad-2541-4d10-9519-7174fe60b998 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:42.063760 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1277/2559] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:21:41 2022] DELETE /identity/v3/projects/618c9b539fbe4c3dbc1ecfb83e47118a => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:42.067767 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc739cc-6cb2-41f2-b269-931e863cc6bc service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:42.073244 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:42.079133 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc739cc-6cb2-41f2-b269-931e863cc6bc service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:42.134449 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92073a78-3353-4678-966d-d00487bf7ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:42.135087 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92073a78-3353-4678-966d-d00487bf7ccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:42.135374 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92073a78-3353-4678-966d-d00487bf7ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:42.135648 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92073a78-3353-4678-966d-d00487bf7ccd admin admin] PATH_INFO: `/v3/users/3ef8e932d5ee49d9884b94f1d6bcf99a` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:42.136758 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92073a78-3353-4678-966d-d00487bf7ccd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ef8e932d5ee49d9884b94f1d6bcf99a)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:42.138676 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92073a78-3353-4678-966d-d00487bf7ccd admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:42.182322 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-92073a78-3353-4678-966d-d00487bf7ccd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ef8e932d5ee49d9884b94f1d6bcf99a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:42.188973 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-92073a78-3353-4678-966d-d00487bf7ccd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ef8e932d5ee49d9884b94f1d6bcf99a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:42.189612 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-92073a78-3353-4678-966d-d00487bf7ccd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ef8e932d5ee49d9884b94f1d6bcf99a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:42.197080 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-92073a78-3353-4678-966d-d00487bf7ccd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ef8e932d5ee49d9884b94f1d6bcf99a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:42.208549 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-92073a78-3353-4678-966d-d00487bf7ccd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ef8e932d5ee49d9884b94f1d6bcf99a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:42.214199 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1283/2560] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:21:42 2022] DELETE /identity/v3/users/3ef8e932d5ee49d9884b94f1d6bcf99a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:42.477853 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c7bad-2541-4d10-9519-7174fe60b998 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:42.479538 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-815c7bad-2541-4d10-9519-7174fe60b998 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:42.493357 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c7bad-2541-4d10-9519-7174fe60b998 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:42.502213 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:42.556286 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a5ff2-4acc-435b-8c01-bf1875f97355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:42.556947 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a5ff2-4acc-435b-8c01-bf1875f97355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:42.557222 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a5ff2-4acc-435b-8c01-bf1875f97355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:42.557494 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a5ff2-4acc-435b-8c01-bf1875f97355 admin admin] PATH_INFO: `/v3/projects/065aa8d6f0e745ca9260f4dcc249f204` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:42.558588 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570a5ff2-4acc-435b-8c01-bf1875f97355 admin admin] RBAC: Authorizing `identity:delete_project(project_id=065aa8d6f0e745ca9260f4dcc249f204)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:42.560871 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570a5ff2-4acc-435b-8c01-bf1875f97355 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:42.604502 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-570a5ff2-4acc-435b-8c01-bf1875f97355 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '065aa8d6f0e745ca9260f4dcc249f204', 'request_id': 'req-570a5ff2-4acc-435b-8c01-bf1875f97355'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:42.609752 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-570a5ff2-4acc-435b-8c01-bf1875f97355 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '065aa8d6f0e745ca9260f4dcc249f204', 'request_id': 'req-570a5ff2-4acc-435b-8c01-bf1875f97355'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:42.615507 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-570a5ff2-4acc-435b-8c01-bf1875f97355 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '065aa8d6f0e745ca9260f4dcc249f204', 'request_id': 'req-570a5ff2-4acc-435b-8c01-bf1875f97355'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:42.628195 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-570a5ff2-4acc-435b-8c01-bf1875f97355 admin admin] The token cache is being invalidate because project 065aa8d6f0e745ca9260f4dcc249f204 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:21:42.628464 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-570a5ff2-4acc-435b-8c01-bf1875f97355 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:42.629076 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-570a5ff2-4acc-435b-8c01-bf1875f97355 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:42.630502 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1278/2561] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:21:42 2022] DELETE /identity/v3/projects/065aa8d6f0e745ca9260f4dcc249f204 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:42.635617 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05f6a1-af98-4f58-bccf-6c7455e715b8 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:42.635901 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05f6a1-af98-4f58-bccf-6c7455e715b8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:42.636172 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05f6a1-af98-4f58-bccf-6c7455e715b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:42.655375 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-cf05f6a1-af98-4f58-bccf-6c7455e715b8 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:21:42.656554 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-cf05f6a1-af98-4f58-bccf-6c7455e715b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:42.713731 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1284/2562] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:21:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:21:42.722890 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa15154-1fbb-430a-859a-c771c4b897e4 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:42.723219 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa15154-1fbb-430a-859a-c771c4b897e4 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:42.723497 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa15154-1fbb-430a-859a-c771c4b897e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:42.743197 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-aaa15154-1fbb-430a-859a-c771c4b897e4 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:21:42.744388 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-aaa15154-1fbb-430a-859a-c771c4b897e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:42.752175 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1279/2563] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:21:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:21:42.756235 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05f6a1-af98-4f58-bccf-6c7455e715b8 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:42.765667 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05f6a1-af98-4f58-bccf-6c7455e715b8 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:42.770346 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44dfd6-9975-496a-8463-a4dff0600f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:42.770963 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44dfd6-9975-496a-8463-a4dff0600f37 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:42.771268 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44dfd6-9975-496a-8463-a4dff0600f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:42.771572 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44dfd6-9975-496a-8463-a4dff0600f37 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:42.772731 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a44dfd6-9975-496a-8463-a4dff0600f37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:21:42.773027 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a44dfd6-9975-496a-8463-a4dff0600f37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:42.774591 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a44dfd6-9975-496a-8463-a4dff0600f37 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:42.785849 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1285/2564] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:21:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:21:42.790601 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa15154-1fbb-430a-859a-c771c4b897e4 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:42.804790 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa15154-1fbb-430a-859a-c771c4b897e4 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:42.810074 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c17aa2-ee93-44ee-9297-ab3945120012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:42.810669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c17aa2-ee93-44ee-9297-ab3945120012 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:42.811001 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c17aa2-ee93-44ee-9297-ab3945120012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:42.811255 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c17aa2-ee93-44ee-9297-ab3945120012 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:42.811883 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c17aa2-ee93-44ee-9297-ab3945120012 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:42.813774 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c17aa2-ee93-44ee-9297-ab3945120012 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:42.840244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1280/2565] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:21:42 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:21:42.844364 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44dfd6-9975-496a-8463-a4dff0600f37 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:42.855110 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44dfd6-9975-496a-8463-a4dff0600f37 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:42.859634 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:42.911979 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d6003-cab6-4831-a46a-9151528765e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:42.912580 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d6003-cab6-4831-a46a-9151528765e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:42.913496 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d6003-cab6-4831-a46a-9151528765e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:42.913762 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d6003-cab6-4831-a46a-9151528765e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:42.914378 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d6003-cab6-4831-a46a-9151528765e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:42.916310 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d6003-cab6-4831-a46a-9151528765e2 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:42.936869 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1286/2566] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:21:42 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:21:42.940913 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c17aa2-ee93-44ee-9297-ab3945120012 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:42.951085 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c17aa2-ee93-44ee-9297-ab3945120012 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:42.955567 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f2fc62-34cd-4f37-8f74-dc794cbf6506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:42.956155 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2fc62-34cd-4f37-8f74-dc794cbf6506 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:42.956431 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2fc62-34cd-4f37-8f74-dc794cbf6506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:42.956700 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2fc62-34cd-4f37-8f74-dc794cbf6506 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:42.957331 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f2fc62-34cd-4f37-8f74-dc794cbf6506 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:42.958945 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f2fc62-34cd-4f37-8f74-dc794cbf6506 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:42.969083 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1281/2567] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:21:42.972740 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d6003-cab6-4831-a46a-9151528765e2 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:42.982867 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d6003-cab6-4831-a46a-9151528765e2 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:42.987491 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e15d8-03af-4cd6-9a75-df475a6c25aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:42.988070 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e15d8-03af-4cd6-9a75-df475a6c25aa admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:42.988342 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e15d8-03af-4cd6-9a75-df475a6c25aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:42.988606 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e15d8-03af-4cd6-9a75-df475a6c25aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:42.989232 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855e15d8-03af-4cd6-9a75-df475a6c25aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:42.990911 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855e15d8-03af-4cd6-9a75-df475a6c25aa admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:43.001066 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1287/2568] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:21:43.005027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f2fc62-34cd-4f37-8f74-dc794cbf6506 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:43.014677 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f2fc62-34cd-4f37-8f74-dc794cbf6506 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:43.019280 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e7aa4-b71d-4eac-bb57-8478a6dab4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:43.019917 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e7aa4-b71d-4eac-bb57-8478a6dab4c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:43.020198 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e7aa4-b71d-4eac-bb57-8478a6dab4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:43.020474 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e7aa4-b71d-4eac-bb57-8478a6dab4c6 admin admin] PATH_INFO: `/v3/projects/a14d5881f8aa4eee98ace42a646b09dc/users/9adcc2d6f1284ea7bab649565d041cfb/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:43.035523 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e7aa4-b71d-4eac-bb57-8478a6dab4c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a14d5881f8aa4eee98ace42a646b09dc, user_id=9adcc2d6f1284ea7bab649565d041cfb, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:43.038766 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e7aa4-b71d-4eac-bb57-8478a6dab4c6 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:43.051388 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1282/2569] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:21:43 2022] PUT /identity/v3/projects/a14d5881f8aa4eee98ace42a646b09dc/users/9adcc2d6f1284ea7bab649565d041cfb/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:43.061600 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9319932-8b8c-45e0-aef0-0fd439425acd None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:43.061819 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9319932-8b8c-45e0-aef0-0fd439425acd None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:43.062084 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9319932-8b8c-45e0-aef0-0fd439425acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:43.089364 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-a9319932-8b8c-45e0-aef0-0fd439425acd None None] MFA Rules not processed for user `9adcc2d6f1284ea7bab649565d041cfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:21:43.090539 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a9319932-8b8c-45e0-aef0-0fd439425acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:43.153511 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1288/2570] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:21:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:21:43.157871 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e7aa4-b71d-4eac-bb57-8478a6dab4c6 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:43.198700 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:43.221091 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e7aa4-b71d-4eac-bb57-8478a6dab4c6 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:43.225696 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5f02d-37d0-4271-8d54-f851b895e993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:43.226288 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5f02d-37d0-4271-8d54-f851b895e993 admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:43.226568 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5f02d-37d0-4271-8d54-f851b895e993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:43.226838 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5f02d-37d0-4271-8d54-f851b895e993 admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:43.227512 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5f02d-37d0-4271-8d54-f851b895e993 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:43.229428 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5f02d-37d0-4271-8d54-f851b895e993 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:43.254141 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1283/2571] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:21:43 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:21:43.258636 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9319932-8b8c-45e0-aef0-0fd439425acd None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:43.269087 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9319932-8b8c-45e0-aef0-0fd439425acd None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:43.322602 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7fecda-4e54-435a-9bc0-440bff231b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:43.323143 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fecda-4e54-435a-9bc0-440bff231b53 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:43.323419 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fecda-4e54-435a-9bc0-440bff231b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:43.323686 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fecda-4e54-435a-9bc0-440bff231b53 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:43.324301 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7fecda-4e54-435a-9bc0-440bff231b53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:43.326314 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7fecda-4e54-435a-9bc0-440bff231b53 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:43.347242 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1289/2572] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:21:43 2022] POST /identity/v3/users => generated 298 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:21:43.351244 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5f02d-37d0-4271-8d54-f851b895e993 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:43.361428 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5f02d-37d0-4271-8d54-f851b895e993 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:43.366260 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc5540-9bc0-4597-8bc5-4b4112c8ae54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:43.366862 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc5540-9bc0-4597-8bc5-4b4112c8ae54 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:43.367194 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc5540-9bc0-4597-8bc5-4b4112c8ae54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:43.367468 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc5540-9bc0-4597-8bc5-4b4112c8ae54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:43.368109 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cc5540-9bc0-4597-8bc5-4b4112c8ae54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:43.369720 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cc5540-9bc0-4597-8bc5-4b4112c8ae54 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:43.380100 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1284/2573] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:21:43.384595 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7fecda-4e54-435a-9bc0-440bff231b53 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:43.394282 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7fecda-4e54-435a-9bc0-440bff231b53 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:43.399652 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803ffd3-9d28-4446-80da-e09d72448cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:43.400629 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803ffd3-9d28-4446-80da-e09d72448cd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:43.400629 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803ffd3-9d28-4446-80da-e09d72448cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:43.400832 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0803ffd3-9d28-4446-80da-e09d72448cd5 admin admin] PATH_INFO: `/v3/projects/c47200fcc18a485191c36c3f025d77e9/users/5f07f65428494211999a63bf348bf693/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:43.403894 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:43.419206 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0803ffd3-9d28-4446-80da-e09d72448cd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c47200fcc18a485191c36c3f025d77e9, user_id=5f07f65428494211999a63bf348bf693, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:43.421935 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0803ffd3-9d28-4446-80da-e09d72448cd5 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:43.434957 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1290/2574] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:21:43 2022] PUT /identity/v3/projects/c47200fcc18a485191c36c3f025d77e9/users/5f07f65428494211999a63bf348bf693/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:43.438876 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc5540-9bc0-4597-8bc5-4b4112c8ae54 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:43.449140 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc5540-9bc0-4597-8bc5-4b4112c8ae54 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:43.501886 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aed609-08e5-4d4d-a71f-dec124ed61cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:43.502473 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aed609-08e5-4d4d-a71f-dec124ed61cf admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:43.502748 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aed609-08e5-4d4d-a71f-dec124ed61cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:43.503015 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aed609-08e5-4d4d-a71f-dec124ed61cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:43.503676 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aed609-08e5-4d4d-a71f-dec124ed61cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:43.505247 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aed609-08e5-4d4d-a71f-dec124ed61cf admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:43.515159 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1285/2575] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:21:43.519276 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803ffd3-9d28-4446-80da-e09d72448cd5 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:43.529645 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0803ffd3-9d28-4446-80da-e09d72448cd5 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:43.534228 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f155ce7-f878-4640-ba93-0c695cdf52bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:43.534867 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f155ce7-f878-4640-ba93-0c695cdf52bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:43.535179 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f155ce7-f878-4640-ba93-0c695cdf52bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:43.535452 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f155ce7-f878-4640-ba93-0c695cdf52bd admin admin] PATH_INFO: `/v3/projects/c47200fcc18a485191c36c3f025d77e9/users/5f07f65428494211999a63bf348bf693/roles/70af94fc6a8149f2a38f74b843e1a9f0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:43.537330 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f155ce7-f878-4640-ba93-0c695cdf52bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c47200fcc18a485191c36c3f025d77e9, user_id=5f07f65428494211999a63bf348bf693, role_id=70af94fc6a8149f2a38f74b843e1a9f0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:43.539984 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f155ce7-f878-4640-ba93-0c695cdf52bd admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:43.550202 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1291/2576] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:21:43 2022] PUT /identity/v3/projects/c47200fcc18a485191c36c3f025d77e9/users/5f07f65428494211999a63bf348bf693/roles/70af94fc6a8149f2a38f74b843e1a9f0 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:43.560805 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934655ad-ca22-48f5-853c-0c5441dacef4 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:43.561173 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934655ad-ca22-48f5-853c-0c5441dacef4 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:43.561449 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934655ad-ca22-48f5-853c-0c5441dacef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:43.588743 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-934655ad-ca22-48f5-853c-0c5441dacef4 None None] MFA Rules not processed for user `5f07f65428494211999a63bf348bf693`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:21:43.589892 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-934655ad-ca22-48f5-853c-0c5441dacef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:43.653860 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1286/2577] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:21:43 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:21:43.664366 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f155ce7-f878-4640-ba93-0c695cdf52bd admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:43.666050 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3f155ce7-f878-4640-ba93-0c695cdf52bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:43.727256 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f155ce7-f878-4640-ba93-0c695cdf52bd admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:43.730870 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6470b5c1-16f4-4e5b-a0d1-540438504ed8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:43.731578 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470b5c1-16f4-4e5b-a0d1-540438504ed8 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:43.731852 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470b5c1-16f4-4e5b-a0d1-540438504ed8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:43.732122 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470b5c1-16f4-4e5b-a0d1-540438504ed8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:43.786806 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6470b5c1-16f4-4e5b-a0d1-540438504ed8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:43.788781 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6470b5c1-16f4-4e5b-a0d1-540438504ed8 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:43.797391 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1292/2578] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:21:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:21:48.167400 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934655ad-ca22-48f5-853c-0c5441dacef4 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:48.168773 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-934655ad-ca22-48f5-853c-0c5441dacef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:48.181501 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934655ad-ca22-48f5-853c-0c5441dacef4 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:48.184996 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11549e12-d2c4-4245-b7be-25745d71b528 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:48.185607 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11549e12-d2c4-4245-b7be-25745d71b528 service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:48.185883 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11549e12-d2c4-4245-b7be-25745d71b528 service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:48.186152 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11549e12-d2c4-4245-b7be-25745d71b528 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:48.195640 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11549e12-d2c4-4245-b7be-25745d71b528 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:48.197064 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11549e12-d2c4-4245-b7be-25745d71b528 service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:48.206450 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1287/2579] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:21:48 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:21:53.090529 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6470b5c1-16f4-4e5b-a0d1-540438504ed8 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:53.151252 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6470b5c1-16f4-4e5b-a0d1-540438504ed8 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:53.155747 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780bbe6-38ba-40f8-a8d9-b2d291864cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:53.156364 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780bbe6-38ba-40f8-a8d9-b2d291864cd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:53.156644 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780bbe6-38ba-40f8-a8d9-b2d291864cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:53.156917 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780bbe6-38ba-40f8-a8d9-b2d291864cd1 admin admin] PATH_INFO: `/v3/users/9adcc2d6f1284ea7bab649565d041cfb` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:53.158021 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780bbe6-38ba-40f8-a8d9-b2d291864cd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9adcc2d6f1284ea7bab649565d041cfb)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:53.159984 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780bbe6-38ba-40f8-a8d9-b2d291864cd1 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:53.187328 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:53.208209 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b780bbe6-38ba-40f8-a8d9-b2d291864cd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9adcc2d6f1284ea7bab649565d041cfb'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:53.215348 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b780bbe6-38ba-40f8-a8d9-b2d291864cd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9adcc2d6f1284ea7bab649565d041cfb'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:53.215998 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b780bbe6-38ba-40f8-a8d9-b2d291864cd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9adcc2d6f1284ea7bab649565d041cfb'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:53.223127 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b780bbe6-38ba-40f8-a8d9-b2d291864cd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9adcc2d6f1284ea7bab649565d041cfb'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:53.235294 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b780bbe6-38ba-40f8-a8d9-b2d291864cd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9adcc2d6f1284ea7bab649565d041cfb'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:53.241233 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1293/2580] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:21:53 2022] DELETE /identity/v3/users/9adcc2d6f1284ea7bab649565d041cfb => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:53.245747 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56684b04-61f9-4857-be95-d6d93880b4cc None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:53.246246 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56684b04-61f9-4857-be95-d6d93880b4cc None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:53.246361 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56684b04-61f9-4857-be95-d6d93880b4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:53.264834 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-56684b04-61f9-4857-be95-d6d93880b4cc None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:21:53.266020 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-56684b04-61f9-4857-be95-d6d93880b4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:53.279107 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:53.324591 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1288/2581] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:21:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:21:53.334167 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780bbe6-38ba-40f8-a8d9-b2d291864cd1 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:53.335744 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b780bbe6-38ba-40f8-a8d9-b2d291864cd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:53.392469 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780bbe6-38ba-40f8-a8d9-b2d291864cd1 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:53.396040 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae76a3-2a2b-43ac-8540-6d74932a3eca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:53.396632 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae76a3-2a2b-43ac-8540-6d74932a3eca service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:53.396934 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae76a3-2a2b-43ac-8540-6d74932a3eca service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:53.397202 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae76a3-2a2b-43ac-8540-6d74932a3eca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:53.409542 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae76a3-2a2b-43ac-8540-6d74932a3eca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:53.411108 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae76a3-2a2b-43ac-8540-6d74932a3eca service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:53.421417 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1294/2582] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:21:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:21:53.683378 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56684b04-61f9-4857-be95-d6d93880b4cc None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:53.685000 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-56684b04-61f9-4857-be95-d6d93880b4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:53.701540 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56684b04-61f9-4857-be95-d6d93880b4cc None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:53.706386 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d9d7e-fc18-4966-b3ae-ae607bf52688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:53.707017 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d9d7e-fc18-4966-b3ae-ae607bf52688 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:53.707327 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d9d7e-fc18-4966-b3ae-ae607bf52688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:53.707598 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d9d7e-fc18-4966-b3ae-ae607bf52688 admin admin] PATH_INFO: `/v3/projects/a14d5881f8aa4eee98ace42a646b09dc` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:53.708880 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01d9d7e-fc18-4966-b3ae-ae607bf52688 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a14d5881f8aa4eee98ace42a646b09dc)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:53.710672 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01d9d7e-fc18-4966-b3ae-ae607bf52688 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:53.756833 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-e01d9d7e-fc18-4966-b3ae-ae607bf52688 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a14d5881f8aa4eee98ace42a646b09dc', 'request_id': 'req-e01d9d7e-fc18-4966-b3ae-ae607bf52688'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:53.762980 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-e01d9d7e-fc18-4966-b3ae-ae607bf52688 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a14d5881f8aa4eee98ace42a646b09dc', 'request_id': 'req-e01d9d7e-fc18-4966-b3ae-ae607bf52688'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:53.769109 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-e01d9d7e-fc18-4966-b3ae-ae607bf52688 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a14d5881f8aa4eee98ace42a646b09dc', 'request_id': 'req-e01d9d7e-fc18-4966-b3ae-ae607bf52688'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:53.782088 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-e01d9d7e-fc18-4966-b3ae-ae607bf52688 admin admin] The token cache is being invalidate because project a14d5881f8aa4eee98ace42a646b09dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:21:53.782373 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-e01d9d7e-fc18-4966-b3ae-ae607bf52688 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:53.782982 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-e01d9d7e-fc18-4966-b3ae-ae607bf52688 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:53.784525 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1289/2583] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:21:53 2022] DELETE /identity/v3/projects/a14d5881f8aa4eee98ace42a646b09dc => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:53.788626 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae76a3-2a2b-43ac-8540-6d74932a3eca service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:53.798053 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae76a3-2a2b-43ac-8540-6d74932a3eca service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:53.852308 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ebbbf-b96a-4e28-8313-ade0e15ee64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:53.852940 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ebbbf-b96a-4e28-8313-ade0e15ee64f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:53.853245 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ebbbf-b96a-4e28-8313-ade0e15ee64f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:53.853511 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ebbbf-b96a-4e28-8313-ade0e15ee64f admin admin] PATH_INFO: `/v3/users/5f07f65428494211999a63bf348bf693` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:53.854639 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65ebbbf-b96a-4e28-8313-ade0e15ee64f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f07f65428494211999a63bf348bf693)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:53.856599 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65ebbbf-b96a-4e28-8313-ade0e15ee64f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:53.900970 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f65ebbbf-b96a-4e28-8313-ade0e15ee64f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f07f65428494211999a63bf348bf693'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:53.902089 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:53.910786 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f65ebbbf-b96a-4e28-8313-ade0e15ee64f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f07f65428494211999a63bf348bf693'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:53.911487 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f65ebbbf-b96a-4e28-8313-ade0e15ee64f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f07f65428494211999a63bf348bf693'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:53.918773 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f65ebbbf-b96a-4e28-8313-ade0e15ee64f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f07f65428494211999a63bf348bf693'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:53.932013 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f65ebbbf-b96a-4e28-8313-ade0e15ee64f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f07f65428494211999a63bf348bf693'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:53.937817 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1295/2584] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:21:53 2022] DELETE /identity/v3/users/5f07f65428494211999a63bf348bf693 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:54.199673 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d9d7e-fc18-4966-b3ae-ae607bf52688 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:54.201194 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-e01d9d7e-fc18-4966-b3ae-ae607bf52688 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:54.215081 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d9d7e-fc18-4966-b3ae-ae607bf52688 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:54.271465 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee9e9d-0485-46d3-b8b5-f0d29dd5ef11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:54.272154 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee9e9d-0485-46d3-b8b5-f0d29dd5ef11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:54.272360 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee9e9d-0485-46d3-b8b5-f0d29dd5ef11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:54.272631 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee9e9d-0485-46d3-b8b5-f0d29dd5ef11 admin admin] PATH_INFO: `/v3/projects/c47200fcc18a485191c36c3f025d77e9` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:54.273771 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee9e9d-0485-46d3-b8b5-f0d29dd5ef11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c47200fcc18a485191c36c3f025d77e9)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:54.275723 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee9e9d-0485-46d3-b8b5-f0d29dd5ef11 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:54.318515 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:54.322015 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c4ee9e9d-0485-46d3-b8b5-f0d29dd5ef11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c47200fcc18a485191c36c3f025d77e9', 'request_id': 'req-c4ee9e9d-0485-46d3-b8b5-f0d29dd5ef11'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:54.328606 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c4ee9e9d-0485-46d3-b8b5-f0d29dd5ef11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c47200fcc18a485191c36c3f025d77e9', 'request_id': 'req-c4ee9e9d-0485-46d3-b8b5-f0d29dd5ef11'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:54.334702 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c4ee9e9d-0485-46d3-b8b5-f0d29dd5ef11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c47200fcc18a485191c36c3f025d77e9', 'request_id': 'req-c4ee9e9d-0485-46d3-b8b5-f0d29dd5ef11'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:54.345662 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c4ee9e9d-0485-46d3-b8b5-f0d29dd5ef11 admin admin] The token cache is being invalidate because project c47200fcc18a485191c36c3f025d77e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:21:54.345951 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c4ee9e9d-0485-46d3-b8b5-f0d29dd5ef11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:54.346582 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c4ee9e9d-0485-46d3-b8b5-f0d29dd5ef11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:54.348073 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1290/2585] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:21:54 2022] DELETE /identity/v3/projects/c47200fcc18a485191c36c3f025d77e9 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:54.354597 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6669d-da12-4117-ba12-4a005e5680f9 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:54.355085 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6669d-da12-4117-ba12-4a005e5680f9 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:54.355204 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6669d-da12-4117-ba12-4a005e5680f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:54.374850 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-15e6669d-da12-4117-ba12-4a005e5680f9 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:21:54.376096 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-15e6669d-da12-4117-ba12-4a005e5680f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:54.432845 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1296/2586] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:21:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:21:54.442425 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0660496-48f1-4eba-a266-6220f0c11898 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:54.442706 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0660496-48f1-4eba-a266-6220f0c11898 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:54.442976 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0660496-48f1-4eba-a266-6220f0c11898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:54.462156 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-f0660496-48f1-4eba-a266-6220f0c11898 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:21:54.463369 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-f0660496-48f1-4eba-a266-6220f0c11898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:54.471043 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1291/2587] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:21:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:21:54.475029 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6669d-da12-4117-ba12-4a005e5680f9 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:54.484917 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6669d-da12-4117-ba12-4a005e5680f9 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:54.489427 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e767c-04ef-4c01-ab0d-7ee071ea63c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:54.490013 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e767c-04ef-4c01-ab0d-7ee071ea63c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:54.490288 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e767c-04ef-4c01-ab0d-7ee071ea63c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:54.490554 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e767c-04ef-4c01-ab0d-7ee071ea63c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:54.491222 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e767c-04ef-4c01-ab0d-7ee071ea63c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 13 13:21:54.491521 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e767c-04ef-4c01-ab0d-7ee071ea63c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:54.493672 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e767c-04ef-4c01-ab0d-7ee071ea63c0 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:54.504753 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1297/2588] 10.222.0.23 () {58 vars in 1153 bytes} [Thu Oct 13 13:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 13 13:21:54.509015 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0660496-48f1-4eba-a266-6220f0c11898 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:54.518276 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0660496-48f1-4eba-a266-6220f0c11898 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:54.523144 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf2d71-da30-414b-8f3a-47531848b6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:54.523730 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf2d71-da30-414b-8f3a-47531848b6aa admin admin] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:54.524008 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf2d71-da30-414b-8f3a-47531848b6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:54.524283 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf2d71-da30-414b-8f3a-47531848b6aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:54.524918 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cf2d71-da30-414b-8f3a-47531848b6aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:54.526822 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cf2d71-da30-414b-8f3a-47531848b6aa admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:54.553490 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1292/2589] 10.222.0.23 () {60 vars in 1153 bytes} [Thu Oct 13 13:21:54 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:21:54.558273 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e767c-04ef-4c01-ab0d-7ee071ea63c0 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:54.567986 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e767c-04ef-4c01-ab0d-7ee071ea63c0 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:54.622367 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585d66d-6357-4c93-82cf-4aebd45c8563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:54.622923 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585d66d-6357-4c93-82cf-4aebd45c8563 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:54.623240 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585d66d-6357-4c93-82cf-4aebd45c8563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:54.623515 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585d66d-6357-4c93-82cf-4aebd45c8563 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:54.624145 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e585d66d-6357-4c93-82cf-4aebd45c8563 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:54.626086 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e585d66d-6357-4c93-82cf-4aebd45c8563 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:54.646445 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1298/2590] 10.222.0.23 () {60 vars in 1144 bytes} [Thu Oct 13 13:21:54 2022] POST /identity/v3/users => generated 293 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 13 13:21:54.651180 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf2d71-da30-414b-8f3a-47531848b6aa admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:54.661316 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf2d71-da30-414b-8f3a-47531848b6aa admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:54.666509 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9bf2e-b064-48c8-811a-268af7cbdd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:54.667129 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9bf2e-b064-48c8-811a-268af7cbdd65 admin admin] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:54.667415 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9bf2e-b064-48c8-811a-268af7cbdd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:54.667687 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9bf2e-b064-48c8-811a-268af7cbdd65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:54.668320 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9bf2e-b064-48c8-811a-268af7cbdd65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:54.669903 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9bf2e-b064-48c8-811a-268af7cbdd65 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:54.679975 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1293/2591] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:21:54.683826 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585d66d-6357-4c93-82cf-4aebd45c8563 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:54.693558 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585d66d-6357-4c93-82cf-4aebd45c8563 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:54.698069 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb66558-3a82-4e02-8a36-9aa95bd0353e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:54.698653 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb66558-3a82-4e02-8a36-9aa95bd0353e admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:54.698961 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb66558-3a82-4e02-8a36-9aa95bd0353e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:54.699267 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb66558-3a82-4e02-8a36-9aa95bd0353e admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:54.699906 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb66558-3a82-4e02-8a36-9aa95bd0353e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:54.701474 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb66558-3a82-4e02-8a36-9aa95bd0353e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:54.713431 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1299/2592] 10.222.0.23 () {58 vars in 1122 bytes} [Thu Oct 13 13:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 13 13:21:54.717527 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9bf2e-b064-48c8-811a-268af7cbdd65 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:54.728030 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9bf2e-b064-48c8-811a-268af7cbdd65 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:54.732589 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30028235-0b9a-459e-9d24-0875015db286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:54.733234 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30028235-0b9a-459e-9d24-0875015db286 admin admin] REQUEST_METHOD: `PUT` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:54.733557 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30028235-0b9a-459e-9d24-0875015db286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:54.733787 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30028235-0b9a-459e-9d24-0875015db286 admin admin] PATH_INFO: `/v3/projects/b650d01f212b4cf59d2fb1e8b67588a5/users/2a8b89e1b1b04238bc1dbb5a925512a3/roles/10ba22bff4d24fd69e4c6def38370617` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:54.749158 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30028235-0b9a-459e-9d24-0875015db286 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b650d01f212b4cf59d2fb1e8b67588a5, user_id=2a8b89e1b1b04238bc1dbb5a925512a3, role_id=10ba22bff4d24fd69e4c6def38370617)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:54.751870 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30028235-0b9a-459e-9d24-0875015db286 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:54.765396 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1294/2593] 10.222.0.23 () {60 vars in 1483 bytes} [Thu Oct 13 13:21:54 2022] PUT /identity/v3/projects/b650d01f212b4cf59d2fb1e8b67588a5/users/2a8b89e1b1b04238bc1dbb5a925512a3/roles/10ba22bff4d24fd69e4c6def38370617 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:54.777237 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81942469-03d4-4b3f-97aa-eaaa771a2665 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:54.777522 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81942469-03d4-4b3f-97aa-eaaa771a2665 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:54.777795 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81942469-03d4-4b3f-97aa-eaaa771a2665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:54.805408 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-81942469-03d4-4b3f-97aa-eaaa771a2665 None None] MFA Rules not processed for user `2a8b89e1b1b04238bc1dbb5a925512a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:21:54.806589 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-81942469-03d4-4b3f-97aa-eaaa771a2665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:54.823976 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:54.872098 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1300/2594] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:21:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:21:54.883587 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30028235-0b9a-459e-9d24-0875015db286 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:54.885247 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-30028235-0b9a-459e-9d24-0875015db286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:54.944461 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30028235-0b9a-459e-9d24-0875015db286 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:54.948101 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07146656-3216-4332-9640-131049b0622c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2398a599165a46adb5718d98c897f238', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:54.948699 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07146656-3216-4332-9640-131049b0622c service cinder] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:54.948979 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07146656-3216-4332-9640-131049b0622c service cinder] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:54.949247 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07146656-3216-4332-9640-131049b0622c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:54.957391 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07146656-3216-4332-9640-131049b0622c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:54.958809 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07146656-3216-4332-9640-131049b0622c service cinder] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:54.968041 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1295/2595] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:21:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:21:57.512507 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81942469-03d4-4b3f-97aa-eaaa771a2665 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:57.573297 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81942469-03d4-4b3f-97aa-eaaa771a2665 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:57.577729 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f4da3-7d9a-4f49-bde1-e73194bbcd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:57.578341 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f4da3-7d9a-4f49-bde1-e73194bbcd70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:57.578617 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f4da3-7d9a-4f49-bde1-e73194bbcd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:57.578886 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f4da3-7d9a-4f49-bde1-e73194bbcd70 admin admin] PATH_INFO: `/v3/users/2a8b89e1b1b04238bc1dbb5a925512a3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:57.580100 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f4da3-7d9a-4f49-bde1-e73194bbcd70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a8b89e1b1b04238bc1dbb5a925512a3)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:57.582032 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f4da3-7d9a-4f49-bde1-e73194bbcd70 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:57.626010 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a29f4da3-7d9a-4f49-bde1-e73194bbcd70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a8b89e1b1b04238bc1dbb5a925512a3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:57.632668 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a29f4da3-7d9a-4f49-bde1-e73194bbcd70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a8b89e1b1b04238bc1dbb5a925512a3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:57.633578 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a29f4da3-7d9a-4f49-bde1-e73194bbcd70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a8b89e1b1b04238bc1dbb5a925512a3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:57.640450 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a29f4da3-7d9a-4f49-bde1-e73194bbcd70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a8b89e1b1b04238bc1dbb5a925512a3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:57.652576 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a29f4da3-7d9a-4f49-bde1-e73194bbcd70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a8b89e1b1b04238bc1dbb5a925512a3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:57.658400 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1301/2596] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:21:57 2022] DELETE /identity/v3/users/2a8b89e1b1b04238bc1dbb5a925512a3 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:21:57.662852 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd74281-19b4-4b64-b790-db794fa73ac6 None None] REQUEST_METHOD: `POST` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:57.663348 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd74281-19b4-4b64-b790-db794fa73ac6 None None] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:57.663462 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd74281-19b4-4b64-b790-db794fa73ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:57.682671 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.auth.core [None req-5fd74281-19b4-4b64-b790-db794fa73ac6 None None] MFA Rules not processed for user `22269fc49ecd49a7b97d86f5937fc2dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81520) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:21:57.683878 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-5fd74281-19b4-4b64-b790-db794fa73ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:57.741246 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1296/2597] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:21:57.750869 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f4da3-7d9a-4f49-bde1-e73194bbcd70 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:57.752296 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a29f4da3-7d9a-4f49-bde1-e73194bbcd70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:57.794991 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:57.810025 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f4da3-7d9a-4f49-bde1-e73194bbcd70 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:57.813637 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a40dff-c53e-4dcf-bb30-abdd87b71fe6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:57.814231 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a40dff-c53e-4dcf-bb30-abdd87b71fe6 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:57.814510 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a40dff-c53e-4dcf-bb30-abdd87b71fe6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:57.814776 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a40dff-c53e-4dcf-bb30-abdd87b71fe6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:57.823578 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a40dff-c53e-4dcf-bb30-abdd87b71fe6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:57.824930 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a40dff-c53e-4dcf-bb30-abdd87b71fe6 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:57.834111 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1302/2598] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:21:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:21:58.095123 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd74281-19b4-4b64-b790-db794fa73ac6 None None] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:21:58.096637 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-5fd74281-19b4-4b64-b790-db794fa73ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:21:58.108417 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd74281-19b4-4b64-b790-db794fa73ac6 None None] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:21:58.112964 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ba99d-050a-4324-926b-79078ca1928c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:21:58.113590 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ba99d-050a-4324-926b-79078ca1928c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:21:58.113867 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ba99d-050a-4324-926b-79078ca1928c admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:21:58.114139 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ba99d-050a-4324-926b-79078ca1928c admin admin] PATH_INFO: `/v3/projects/b650d01f212b4cf59d2fb1e8b67588a5` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:21:58.115277 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419ba99d-050a-4324-926b-79078ca1928c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b650d01f212b4cf59d2fb1e8b67588a5)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:21:58.117178 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419ba99d-050a-4324-926b-79078ca1928c admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:21:58.125279 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:21:58.165448 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-419ba99d-050a-4324-926b-79078ca1928c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b650d01f212b4cf59d2fb1e8b67588a5', 'request_id': 'req-419ba99d-050a-4324-926b-79078ca1928c'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:58.170690 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-419ba99d-050a-4324-926b-79078ca1928c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b650d01f212b4cf59d2fb1e8b67588a5', 'request_id': 'req-419ba99d-050a-4324-926b-79078ca1928c'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:58.177005 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-419ba99d-050a-4324-926b-79078ca1928c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b650d01f212b4cf59d2fb1e8b67588a5', 'request_id': 'req-419ba99d-050a-4324-926b-79078ca1928c'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:58.188630 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-419ba99d-050a-4324-926b-79078ca1928c admin admin] The token cache is being invalidate because project b650d01f212b4cf59d2fb1e8b67588a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81520) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:21:58.188915 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-419ba99d-050a-4324-926b-79078ca1928c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:58.189530 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-419ba99d-050a-4324-926b-79078ca1928c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:21:58.190997 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1297/2599] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:21:58 2022] DELETE /identity/v3/projects/b650d01f212b4cf59d2fb1e8b67588a5 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:22:07.829940 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=8 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:22:08.185687 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=21,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:22:18.224071 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc84c09-ef5a-4b6b-8443-4601ba213bac None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:22:18.224564 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc84c09-ef5a-4b6b-8443-4601ba213bac None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:22:18.224665 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc84c09-ef5a-4b6b-8443-4601ba213bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:22:18.244528 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-dbc84c09-ef5a-4b6b-8443-4601ba213bac None None] MFA Rules not processed for user `cd005dcc25944dfc9aeda1eb1468a0ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 13 13:22:18.246321 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-dbc84c09-ef5a-4b6b-8443-4601ba213bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:22:18.298424 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1303/2600] 10.222.0.23 () {58 vars in 958 bytes} [Thu Oct 13 13:22:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 13 13:22:18.619323 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ba99d-050a-4324-926b-79078ca1928c admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:22:18.620726 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-419ba99d-050a-4324-926b-79078ca1928c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:22:18.675789 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ba99d-050a-4324-926b-79078ca1928c admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:22:18.679352 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbbad7-0e85-44ab-af70-9c1d49c641d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:22:18.679885 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbbad7-0e85-44ab-af70-9c1d49c641d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:22:18.680158 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbbad7-0e85-44ab-af70-9c1d49c641d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:22:18.680426 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbbad7-0e85-44ab-af70-9c1d49c641d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:22:18.682086 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-58cbbad7-0e85-44ab-af70-9c1d49c641d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:22:18.691202 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cbbad7-0e85-44ab-af70-9c1d49c641d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:22:18.692605 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cbbad7-0e85-44ab-af70-9c1d49c641d7 service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:22:18.700485 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1298/2601] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:22:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:22:28.291532 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=25 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:22:28.696826 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=26 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:22:36.305777 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc84c09-ef5a-4b6b-8443-4601ba213bac None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:22:36.307232 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-dbc84c09-ef5a-4b6b-8443-4601ba213bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:22:36.381813 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc84c09-ef5a-4b6b-8443-4601ba213bac None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:22:36.387876 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac1f21-3387-44a5-aa72-5bdbfd69c39b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12700d985a724595af630c81e63f86d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:22:36.388514 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac1f21-3387-44a5-aa72-5bdbfd69c39b service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:22:36.388794 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac1f21-3387-44a5-aa72-5bdbfd69c39b service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:22:36.389064 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac1f21-3387-44a5-aa72-5bdbfd69c39b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:22:36.390817 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-01ac1f21-3387-44a5-aa72-5bdbfd69c39b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:22:36.403762 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac1f21-3387-44a5-aa72-5bdbfd69c39b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:22:36.405336 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac1f21-3387-44a5-aa72-5bdbfd69c39b service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:22:36.416085 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1304/2602] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:22:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:22:46.411444 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=34 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:23:03.053403 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbbad7-0e85-44ab-af70-9c1d49c641d7 service neutron] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:23:03.054803 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-58cbbad7-0e85-44ab-af70-9c1d49c641d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:23:03.121597 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbbad7-0e85-44ab-af70-9c1d49c641d7 service neutron] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:23:03.126432 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfdf890-2447-4db8-8987-fbab3db0a07f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a971cdb7f9d4e6692c6326c103d7303', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:23:03.127029 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfdf890-2447-4db8-8987-fbab3db0a07f service placement] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:23:03.127356 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfdf890-2447-4db8-8987-fbab3db0a07f service placement] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:23:03.127627 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfdf890-2447-4db8-8987-fbab3db0a07f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:23:03.129312 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-6dfdf890-2447-4db8-8987-fbab3db0a07f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:23:03.140377 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfdf890-2447-4db8-8987-fbab3db0a07f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:23:03.141943 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfdf890-2447-4db8-8987-fbab3db0a07f service placement] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:23:03.151764 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1299/2603] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:23:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:23:13.147445 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=32 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:24:15.172233 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac1f21-3387-44a5-aa72-5bdbfd69c39b service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:24:15.181805 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac1f21-3387-44a5-aa72-5bdbfd69c39b service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:24:15.185346 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b8f1d-df54-4cd5-8792-88f5f6277219 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:24:15.185924 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b8f1d-df54-4cd5-8792-88f5f6277219 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:24:15.186196 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b8f1d-df54-4cd5-8792-88f5f6277219 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:24:15.186460 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b8f1d-df54-4cd5-8792-88f5f6277219 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:24:15.188188 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-330b8f1d-df54-4cd5-8792-88f5f6277219 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:24:15.198437 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330b8f1d-df54-4cd5-8792-88f5f6277219 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:24:15.199860 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330b8f1d-df54-4cd5-8792-88f5f6277219 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:24:15.208330 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1305/2604] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:24:15 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:24:25.204525 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=6 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:24:28.207489 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfdf890-2447-4db8-8987-fbab3db0a07f service placement] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:24:28.209027 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-6dfdf890-2447-4db8-8987-fbab3db0a07f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:24:28.221260 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfdf890-2447-4db8-8987-fbab3db0a07f service placement] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:24:28.225044 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51585390-cd6b-4bf6-a87e-df30e011619b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '846251a661c04d3f9f195c4ac09b4189', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:24:28.225646 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51585390-cd6b-4bf6-a87e-df30e011619b service neutron] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:24:28.225924 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51585390-cd6b-4bf6-a87e-df30e011619b service neutron] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:24:28.226225 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51585390-cd6b-4bf6-a87e-df30e011619b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:24:28.227960 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-51585390-cd6b-4bf6-a87e-df30e011619b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:24:28.289843 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51585390-cd6b-4bf6-a87e-df30e011619b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:24:28.291307 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51585390-cd6b-4bf6-a87e-df30e011619b service neutron] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:24:28.301063 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1300/2605] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:24:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:24:34.486124 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b8f1d-df54-4cd5-8792-88f5f6277219 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:24:34.487542 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-330b8f1d-df54-4cd5-8792-88f5f6277219 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:24:34.502857 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b8f1d-df54-4cd5-8792-88f5f6277219 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:24:34.507694 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff4f78e-ce76-479f-8c75-806eeb06115d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a971cdb7f9d4e6692c6326c103d7303', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:24:34.508285 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4f78e-ce76-479f-8c75-806eeb06115d service placement] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:24:34.508666 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4f78e-ce76-479f-8c75-806eeb06115d service placement] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:24:34.508948 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4f78e-ce76-479f-8c75-806eeb06115d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:24:34.510595 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-bff4f78e-ce76-479f-8c75-806eeb06115d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:24:34.521771 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff4f78e-ce76-479f-8c75-806eeb06115d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:24:34.523972 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff4f78e-ce76-479f-8c75-806eeb06115d service placement] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:24:34.537022 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1306/2606] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:24:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:24:34.550148 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51585390-cd6b-4bf6-a87e-df30e011619b service neutron] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:24:34.552080 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-51585390-cd6b-4bf6-a87e-df30e011619b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:24:34.566127 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51585390-cd6b-4bf6-a87e-df30e011619b service neutron] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:24:34.570751 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958f372-2c23-4030-a1fc-731b50fc7dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:24:34.571426 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958f372-2c23-4030-a1fc-731b50fc7dd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:24:34.571701 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958f372-2c23-4030-a1fc-731b50fc7dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:24:34.571979 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958f372-2c23-4030-a1fc-731b50fc7dd8 admin admin] PATH_INFO: `/v3/users/cd005dcc25944dfc9aeda1eb1468a0ea` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:24:34.573326 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958f372-2c23-4030-a1fc-731b50fc7dd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd005dcc25944dfc9aeda1eb1468a0ea)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:24:34.575343 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958f372-2c23-4030-a1fc-731b50fc7dd8 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:24:34.620145 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-7958f372-2c23-4030-a1fc-731b50fc7dd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd005dcc25944dfc9aeda1eb1468a0ea'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:24:34.626762 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-7958f372-2c23-4030-a1fc-731b50fc7dd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd005dcc25944dfc9aeda1eb1468a0ea'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:24:34.627426 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-7958f372-2c23-4030-a1fc-731b50fc7dd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd005dcc25944dfc9aeda1eb1468a0ea'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:24:34.628018 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-7958f372-2c23-4030-a1fc-731b50fc7dd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd005dcc25944dfc9aeda1eb1468a0ea'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:24:34.635857 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-7958f372-2c23-4030-a1fc-731b50fc7dd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd005dcc25944dfc9aeda1eb1468a0ea'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:24:34.653049 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1301/2607] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:24:34 2022] DELETE /identity/v3/users/cd005dcc25944dfc9aeda1eb1468a0ea => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:24:34.823239 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff4f78e-ce76-479f-8c75-806eeb06115d service placement] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:24:34.834054 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff4f78e-ce76-479f-8c75-806eeb06115d service placement] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:24:34.889508 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b59ea8d-5acf-4322-b3f6-223cee7c55d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:24:34.890053 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59ea8d-5acf-4322-b3f6-223cee7c55d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:24:34.890328 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59ea8d-5acf-4322-b3f6-223cee7c55d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:24:34.890596 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b59ea8d-5acf-4322-b3f6-223cee7c55d6 admin admin] PATH_INFO: `/v3/projects/17e1654782ac4100b0353d561496db51` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:24:34.891768 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b59ea8d-5acf-4322-b3f6-223cee7c55d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17e1654782ac4100b0353d561496db51)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:24:34.893634 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b59ea8d-5acf-4322-b3f6-223cee7c55d6 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:24:34.938966 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-9b59ea8d-5acf-4322-b3f6-223cee7c55d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17e1654782ac4100b0353d561496db51', 'request_id': 'req-9b59ea8d-5acf-4322-b3f6-223cee7c55d6'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:24:34.945397 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-9b59ea8d-5acf-4322-b3f6-223cee7c55d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17e1654782ac4100b0353d561496db51', 'request_id': 'req-9b59ea8d-5acf-4322-b3f6-223cee7c55d6'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:24:34.951383 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-9b59ea8d-5acf-4322-b3f6-223cee7c55d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17e1654782ac4100b0353d561496db51', 'request_id': 'req-9b59ea8d-5acf-4322-b3f6-223cee7c55d6'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:24:34.963987 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-9b59ea8d-5acf-4322-b3f6-223cee7c55d6 admin admin] The token cache is being invalidate because project 17e1654782ac4100b0353d561496db51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:24:34.964272 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-9b59ea8d-5acf-4322-b3f6-223cee7c55d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:24:34.964895 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-9b59ea8d-5acf-4322-b3f6-223cee7c55d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:24:34.966336 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1307/2608] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:24:34 2022] DELETE /identity/v3/projects/17e1654782ac4100b0353d561496db51 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:24:34.970353 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958f372-2c23-4030-a1fc-731b50fc7dd8 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:24:34.972001 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-7958f372-2c23-4030-a1fc-731b50fc7dd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:24:34.984785 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958f372-2c23-4030-a1fc-731b50fc7dd8 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:24:35.038669 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15643ac-c92b-414a-b835-9c9a7cc6c115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:24:35.039306 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15643ac-c92b-414a-b835-9c9a7cc6c115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:24:35.039584 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15643ac-c92b-414a-b835-9c9a7cc6c115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:24:35.039850 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15643ac-c92b-414a-b835-9c9a7cc6c115 admin admin] PATH_INFO: `/v3/users/9eef02a3b7b642e78d13fbb9d6f036c1` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:24:35.040960 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15643ac-c92b-414a-b835-9c9a7cc6c115 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eef02a3b7b642e78d13fbb9d6f036c1)` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:24:35.042861 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15643ac-c92b-414a-b835-9c9a7cc6c115 admin admin] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:24:35.047996 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:24:35.089659 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c15643ac-c92b-414a-b835-9c9a7cc6c115 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eef02a3b7b642e78d13fbb9d6f036c1'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:24:35.096388 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c15643ac-c92b-414a-b835-9c9a7cc6c115 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eef02a3b7b642e78d13fbb9d6f036c1'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:24:35.097037 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c15643ac-c92b-414a-b835-9c9a7cc6c115 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eef02a3b7b642e78d13fbb9d6f036c1'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:24:35.097667 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c15643ac-c92b-414a-b835-9c9a7cc6c115 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eef02a3b7b642e78d13fbb9d6f036c1'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:24:35.104083 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.notifications [None req-c15643ac-c92b-414a-b835-9c9a7cc6c115 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eef02a3b7b642e78d13fbb9d6f036c1'} {{(pid=81520) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:24:35.120885 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1302/2609] 10.222.0.23 () {58 vars in 1224 bytes} [Thu Oct 13 13:24:34 2022] DELETE /identity/v3/users/9eef02a3b7b642e78d13fbb9d6f036c1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:24:35.291583 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b59ea8d-5acf-4322-b3f6-223cee7c55d6 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:24:35.292999 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9b59ea8d-5acf-4322-b3f6-223cee7c55d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:24:35.306321 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b59ea8d-5acf-4322-b3f6-223cee7c55d6 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:24:35.362324 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24638fba-6093-408a-8198-8010d6d2cd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22269fc49ecd49a7b97d86f5937fc2dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9cb9d18ca96e4beea199d190db853658', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:24:35.362959 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24638fba-6093-408a-8198-8010d6d2cd3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:24:35.363395 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24638fba-6093-408a-8198-8010d6d2cd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:24:35.363668 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24638fba-6093-408a-8198-8010d6d2cd3d admin admin] PATH_INFO: `/v3/projects/ad322d9f495a4b33bd84411d7f42df3e` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:24:35.364796 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24638fba-6093-408a-8198-8010d6d2cd3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad322d9f495a4b33bd84411d7f42df3e)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:24:35.367279 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24638fba-6093-408a-8198-8010d6d2cd3d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:24:35.390204 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:24:35.420594 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-24638fba-6093-408a-8198-8010d6d2cd3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad322d9f495a4b33bd84411d7f42df3e', 'request_id': 'req-24638fba-6093-408a-8198-8010d6d2cd3d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:24:35.426325 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-24638fba-6093-408a-8198-8010d6d2cd3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad322d9f495a4b33bd84411d7f42df3e', 'request_id': 'req-24638fba-6093-408a-8198-8010d6d2cd3d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:24:35.433119 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-24638fba-6093-408a-8198-8010d6d2cd3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad322d9f495a4b33bd84411d7f42df3e', 'request_id': 'req-24638fba-6093-408a-8198-8010d6d2cd3d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:24:35.445706 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-24638fba-6093-408a-8198-8010d6d2cd3d admin admin] The token cache is being invalidate because project ad322d9f495a4b33bd84411d7f42df3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 13 13:24:35.445996 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-24638fba-6093-408a-8198-8010d6d2cd3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:24:35.446629 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-24638fba-6093-408a-8198-8010d6d2cd3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 13 13:24:35.448222 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1308/2610] 10.222.0.23 () {58 vars in 1233 bytes} [Thu Oct 13 13:24:35 2022] DELETE /identity/v3/projects/ad322d9f495a4b33bd84411d7f42df3e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 13 13:24:45.117432 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:24:45.442260 dv-cinder-826607-8-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 13 13:28:07.975609 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15643ac-c92b-414a-b835-9c9a7cc6c115 admin admin] Authenticating user token {{(pid=81520) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 13 13:28:07.977006 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-c15643ac-c92b-414a-b835-9c9a7cc6c115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:28:08.116928 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15643ac-c92b-414a-b835-9c9a7cc6c115 admin admin] Validating token access rules against request {{(pid=81520) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 13 13:28:08.174710 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a9048-e83d-4a45-b1e8-c8c4e441880f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a971cdb7f9d4e6692c6326c103d7303', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02308892635844b088ecdcb5413a21a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81520) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 13 13:28:08.175267 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a9048-e83d-4a45-b1e8-c8c4e441880f service placement] REQUEST_METHOD: `GET` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 13 13:28:08.175545 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a9048-e83d-4a45-b1e8-c8c4e441880f service placement] SCRIPT_NAME: `/identity` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 13 13:28:08.175814 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a9048-e83d-4a45-b1e8-c8c4e441880f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81520) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 13 13:28:08.177566 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.fernet_utils [None req-718a9048-e83d-4a45-b1e8-c8c4e441880f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81520) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 13 13:28:08.189552 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a9048-e83d-4a45-b1e8-c8c4e441880f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 13 13:28:08.191174 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a9048-e83d-4a45-b1e8-c8c4e441880f service placement] RBAC: Authorization granted {{(pid=81520) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 13 13:28:08.200880 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: [pid: 81520|app: 0|req: 1303/2611] 10.222.0.23 () {60 vars in 1365 bytes} [Thu Oct 13 13:28:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 13 13:28:18.196715 dv-cinder-826607-8-iscsi devstack@keystone.service[81520]: DEBUG dbcounter [-] [81520] Writing DB stats keystone:SELECT=89 {{(pid=81520) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}